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PREFACE 


One of the major portions of the analytical procedure used by 
NASA— JSC in determining crop acreage is the method used for estimating 
crop proportions. One of the major tasks of this contract was to 
evaluate a set of five potential crop proportion estimators. Several 
studies of these proportion estimators are contained in this annual 
report including an empirical comparison of the different estimators using 
actual data and also an empirical study on the sensitivity (robustness) 
of one class of these estimators, referred to as the class of mixture 
estimators. 

A concern when constructing a practical procedure for estimating 
crop production is the problem of encountering missing data, primarily 
due to cloud cover on one or more passes of the earth observation 
satellite. The effect of missing data upon the crop classification- 
procedures is discussed in detail including a simulation of this missing 
data effect. 

A basic discussion of the potential methods of crop acreage (pro- 
portion) estimation is contained in Paper 1, A literature study on 
these various estimators is contained in the report along with a complete 
description of each estimator including their known properties, bias, 
and MSE (mean square error) . Paper 2 contains an empirical comparison 
of these proportion estimators using actual ERTS-A four channel multi- 
spectral scanner data taken over a 14 square mile test area site in Hill 
County, Montana. The report contains the results of several experiments 
including (for each experiment) the MSB’s for each proportion estimator. 
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One such proportion estimator, the Odell-Chhikara (0-C) estimator, 
cannot be evaluated as previously proposed in earlier reports if the 
confusion matrix used to estimate the population proportions is singular,- 
Paper 3 removes this problem by demonstrating two different methods for 
evaluating the proportion estimate utilizing matrix pseudoinverses and 
a modified Simplex procedure. The MSB for this estimator is also derived. 

Paper 4 is a simulated sensitivity study of one class of these 
proportion estimators, the mixture estimators. Graphs and MSB's are 
included for each of the mixture proportion estimators for each simulated 
experiment. Paper 5 contains a technique for determining one such mix- 
ture proportion estimator, the- maximum likelihood mixture estimator. The 
solution to the normal equations for determining the maximum • likelihood 
estimates is an Iterative one requiring initial estimates. This report 
compares the number of iterations necessary for convergence using 
different initial estimates on a set of actual ERTS data. 

As mentioned previously, a major problem in any workable crop 
production estimation procedure is the problem of missing data. One 
such question that must be answered is "what happens to the quality of 
the proposed classification schemes when one encounters cloud cover on 
a pass of the satellite during one of the biological phases of the crops 
in, some particular region?" In fact, this raises the general question 
of how to classify a region when one encounters missing data. For example, 
one could classify using only the complete data that exists or possibly 
estimate the missing values in some optimum fashion and proceed as if 
the estimated values were actual values. ' These and other methods of 


treating missing data are discussed at length in Paper 6 which includes 



a comparison of the .different methods using different sets of simulated 
data. Assuming a multivariate normal for the distribution of the multi- 
spectral scanner measurements. Paper 7 develops expressions for the 
estimators of the mean vector and covariance matrix using both complete 
and incomplete data. The report also contains a maximum likelihood 
scheme for classifying an observation vector into one of two multivariate 
populations with unknown means but known covariance matrices. Finally, 
Paper 8 considers the a priori probabilities of encountering missing 
data and calculates (and plots) the various probabilities of mis- 
classification for the two population case. 

The final problem addressed in this report is the problem of taking 
yield data (bushels per acre) gathered at several yield stations and 
extrapolating these values over some specified large region. For example, 
one might have such yield data at ten locations (stations) in the state . 
of Kansas and wish to derive a yield estimate at each grid point (for 
some pre-determined grid) lying within the state. Paper 9 examines 
ten such methods ranging from merely using the sample mean to estimate 
the yield at every grid point to much more sophisticated techniques 
requiring extensive computer programming. The report also compares 
these ten extrapolation procedures using an empirical study with, five- 
years of wheat data from North Dakota covering the years 1962-1966. The 
authors also wrote a Fortran contour mapping program that plots the 
extrapolated results of any one method as a contour map of the entire 
region of interest. A contour is defined to be a line (or area) of 
roughly constant yield (i.e. lying within some specified interval). 

The effects of using one extrapolation method over another can 



readily be determined by comparing the corresponding contour maps. 

The final report, Paper 10, is a description of the Fortran com- 
puter programs written by the University of Texas at Dallas in support 
of some of the research activities carried out in these reports. The 
descriptions are very brief and are not meant to be program users’ 
guides but Instead are intended to acquaint the reader with their basic 
intent and input requirements. Further program documentation and computer 
card decks are available upon request. 
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1. Introduction 

In many areas of application of statistical data analysis unlabelled 
observations (observations of unknown classification) are available from 
several homogeneous populations constituting a single heterogeneous popu- 
lation and on the basis of these observations along with varying amount of 
information regarding these homogeneous sub populations one has to estimate 
the proportions and sometimes the number and distribution parameters of 
these subpopulations. For example, in crop acreage estimation problem 
unlabelled observations, sometimes along with some labelled observations 
and perhaps some information about the distribution of individual crop 
populations, from several crops are available and on the basis of such in- 
formation one has to estimate the acreage of a particular crop of interest 
or of all crops as proportion or proportions of total crop acreage. 

The general statistical problem therein can be formulated as follows. 

Let 

^ = {F(x,d) : eeficE } (1) 

m 

be a family of p-dimensional distribution functions, 6 denoting a vector of m 

parameters belonging to a subset Q of the m-dimensional Euclidean space E , 

in 

and G(0) any probability distribution function on fl. Then the distribution 
function 

H(x) - ^ F(x,@) dG(0) (2) 

is called a mixture on ^ with mixing distribution G(0) . When Q - (0 ;|l» **» ®n^ 

is a finite set, then a probability distribution on can be described by a 
finite set G == {aj a 2 ,.., where 

= P(0 = e^). 


(3) 



( 4 ) 


and a mixture H(x) can be obtained as 

H(x) = F(x, e^) , Zui = 1, 

a convex combination of a finite number of distinct elements of 3 *. The 
function tt(x) is now called a finite mixture and the set G a finite TTn'-x-inyr 
distr ibution . The probabilities ot^ (i— N) given by , (3) can also be in- 
terpreted as prior probabilities of F(x,0^)’s. The class H of all finite 
mixtures on is said to be identifiable (Teicher, 1960, 1961; Yakowitz, 

19 69), or, equivalen tly, the class 3* of finite mixing distributions are said 
to be identifiable (Chandra, 1969) if 

|^«i (F(x,0^) = 2 F(x,e') 

J <J J 

implies that N = N*’ and for each i, l£i^, there is some j ,, 1^^^, such that 
Ui = Uj and ^ sample from H(x) is a set of observations {x^,.., x^^,..} 

on random vectors whose distribution functions F(x,©) constitutes H(x) accord- 
ing to (2) or (4). Given a sample x^,,,, x^^ from H(x) and the parametric 
family of distribution functions to which the component distribution function 


of H(x) defined by (4) belong, the identification problem in its most general 
form is the problem of determining the number N, the mixing distribution G 
consisting of the proportions and the parameter vector 0^,»., 0^. 

When the parametric family of distribution functions of the homogeneous 
subpopulation of crops is known, the crop acreage estimation problem can be 
viewed as a special case of identification problem. It is well known (Robbins, 
1964) that the proportions the parameter vectors 9^^,.., and the 

number N of subpopulations are estimable if, and only if, the class H of finite 
mixtures. of distribution functions of subpopulations are identifiable. When 
the parametric family of the distribution functions of the subpopulations are 
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not known the acreage estimation problem then consists of estimating the pro- 
portions otjj and sometimes the number N of subpopulations. In this case 

also, estimation is possible if, and only if, the unknown subpopulation distribu- 
tion functions define a class H of identifiable finite mixtures. Teicher. 

(1960, 1961, 1963) and Yakowitz (1969) have proved that normal (multivariate 
a^ imlya riate ) , binomial^ Poisson families of distribution functi ons ar e 
some of the families of distribution function, the class of finite mixtures 
on which are identifiable. In remote sensing data analysis, the mixtures are 
always asstmied to be identifiable. So, the proportions of the subpopulations 
are always estimable. 

When the number N of subpopulations in a mixture is unknown, Yakowitz (1969) 
has suggested a method of estimation for G and N using Levy distance between 
two distribution functions. But computation of Levy distance between multi- 
variate distribution functions including even multivariate normal distribution 
functions is extremely difficult, practically impossible. Therefore, Yakowitz ’s 
method of estimation, the only available method of estimation in this case, 
is of no practical use. 

In deriving all other estimators, so far mentioned in literature, the 
number N of subpopulations -has been assumed to be known.’ Assuming N to be 
known, we have described some of the available methods of estimation for the 
mixing distribution G. Using the concept of statistically equivalent blocks 
(Tukey, 1947; Wilks,, 1962) we have proposed some new estimators for the mixing 
distribution of a mixture of multivariate distributions. We have excluded 
graphical or semlgraphical techniques (Harding, 1949, Cassie, 1954, Bliscke, 

1964 and Bhattacharya, 1966) used in case of mixtures of univariate distri- 
butions, because, as Day (1969) has observed, these are difficult to extend to 
higher dimensions and appear to have poor sampling properties. 
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I Mo Training Sample Available 
2. Moment Estimators 

The moment estimators were first introduced by Karl Pearson (1894) in 
an' attempt to estimate for a mixture of two univariate normal populations the 
means, variances and proportions by equating the first five moments with their 
sample values. Solving these five equations in the five unknown parameters 
leads to a ninth degree pol 3 momial equation having at least one real root, eac' 
real root giving a set of estimates for the parameters. Pearson proposed that 
the set having its sixth moment nearest the sample sixth moment be used as the 
final estimate. 

Rao (1948) simplified the method of solution assuming the two univariate 

normal distributions in the mixture to have equal variances.. Assume 

H(x) fa F^(x) + (1-a) (5) 

where F^, F^ are univariate normal distributions with mean y^, y^ and common 
2 

variance a . Let and s^ be the second, third and fourth sample moments 

about the mean, and s^^ the first sample moment about the origin for a sample 
obtained from H. Equating these to the corresponding population moments in- 
troduces bias in the estimating equations for a. Hence, equating the first 
four k-statistics of Fisher to their expected values, which are the cumulants 
of H, avoids the bias. The first four k-statistics are given in terms of the 
sample moments by: 
k, = s. 


_ n 


'2 iPT “2 
2 


^3 ~ (n-1) (n-2) ®3 
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Equating these to the corresponding ciomulants yields the estimating 
equations; 

k-i = a + (l-a)u2 
k2 = cT^ + ad^ + (1-a) d^, 
k 3 = adj + (1-cx) d 2 

k^ - adj + (1-a) d^ - 3 [ad^ + (1-a) d^]^ 
where <13^ = Vij_ - k^, and d2 = ^2 “ 

Letting x = value of x is obtained as the negative root of the 

cubic 

X + — k^x + Y k^ = 0. 

If X is the required root, then d^ is given by the negative root of the 
quadratic 

2 ^3 

d, + — d, + X = 0 
1x1 

and d2 by - (“) - d^. .The estimates y2> a and are given by 


A 



= k^ + 


^1 "^2 


A 

a 


2- 


(dj-di) 


''2 

a = k2 + X. 


The fundamental cubic equation ' 

x^ + I k^- X + I k^ - 0 

2 

has a single negative root greater than -k2* Since the coefficient of x is 
absent it is readily obtained. 

The expressions for the standard errors of y^, y2> “ are very 

complicated, but it appears that the estimate of a will have the highest 
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percentage of error whereas the estimates of ]x^ and a will be fairly 
reliable in large samples. 

Using multivariate analogue of k-statistics Day (1969) has extended Rao's 

method to mixtures of two multivariate normals with common covariance matrix. 

2 

In the p-variate case, with p /2 + 5p/2 + 1 parameters to estimate, there are p 
first moments, p(p+l)/2 second moments, p + p(p-l) + p(p-l) (p-2)/6 third 
moments and p + 3 p'(p-l)/2 + p(p-^l) (p-2) /2 + p(p-l) (p-2) (p-3) /24 fourth moments. 
When p>l, since not all third or fourth moments are functionally independent, 
a choice has to be made as to which third and fourth moments, or functions of 
these moments, to use to obtain moment estimates. 

When the covariance metrics of the two multivariate normal distributions 
forming the mixture are unequal, Martin (1936) observed that the multivariate 
analogue of Pearson’s moment equations failed to provide any useful estimate. 

Kabir (1968) considered moment estimates of the mixing distribution for a 
finite (more than two component) mixture on exponential families of distribu- 
tions. It is believed that these estimates are of little practical use. 

3. Maximum Likelihood Estimators 

The maximum likelihood method has also been used in determining estimates 

Vi 

of a and 9^,6„, . . . ,0jj in the mixture H(x) = J)' a^F^(x). The pro- 

1 2 N 

cedure is to determine values of 0 ^, 02 .cij^ and 0 j^, 62 > • • • maximize 

the log of the likelihood function 

n N 

L = £n -H (2 ct.F. (x. ;0j) 

3=1 i=l J 

There are (N-1) a's and 2N 0's to estimate in the case when Fj^,...,Fj^ 

2 

are univariate normal distributions with 0^ = (p^,a^). The estimates can be 
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obtained by solving the system 


9L 

9a 


(6) 

9L 

90 

(g,6j^,..., — 0 . 



This system is nonlinear, but can be solved by known iterative techniques such 
as the Newton-Raphson or gradient method, Hasselblad (1966) has considered the 
maximum likelihood approach, in the case of grouped data. The asymptotic vari- 
ances for the estimates of the parameters were calculated and plotted. The 
asymptotic variances can be obtained from the diagonal elements of -H ^ where 
H is the matrix 


H = 


"<l?> 

®Sa91' 

N-lxK-1 

N-lx2N 



2NxN-l 

2Nx2N 


(7) 


and 


8^L . , 3 ^ L . 

9^ ~^3a^9aj ’ 9?2’ 


9^ 




9^L 

^ *^9y,9o2 » • • • . ay 

~ 1 1 N N 


2 

9^L 


9ajj_l9a 


1 -) 

N 


The expected values in (7) are evaluated at the estimated parameter values. It 
is very difficult, if not impossible, to determine the asymptotic variances of 
the estimates analytically. The variances can, however, be obtained numerically. 

Hasselblad (1969) considered finite mixtures of Poisson, Binomial and ex- 
ponential distribution in a later study by means of maximum likelihood method. 
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Day (1969) considered the method of maximum likelihood in estimating the 
parameters of the mixttire 


H(x) = aF^(x) + (l-o) F^Cx), 

where F^ and F 2 are multivariate normal distributions with pxl vector of means 
and P 2 > respectively and common covariance matrix S. In this case the log 
likelihood function for a sample of size n is 

h(Pi, ^2* ln[(27r)“^P/^|ll“’^/2^ j[ )j 

i=l ^ ^ 


+ (1-a) exp [ (X^ 


( 8 ) 


Taking logarithm and derivatives of L with respect to a, and X, we obtain 


n ®li ®21 


1=1 + (1-a) 62^ 


= 0 


i=l ae^^+(l-a)e2j^ 


= 0 


n 

I 

i=l 


fie, . + (l-a)e 


2i 


21 


^ n 
— n S + S 
i=l 




-1 


= 0 


where 

a ® 

If P(k[x, ) denotes the probability that observation X arises from population 

j 

k, then 


p(ljx^) = a e^7(a e^j + ( 1 -a) e 2 J 

P(2|XJ = 1 - P(l[Xj) 
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The m.£. equations can then be written in the form 


n . n , 

P, ={Z X P(1|X )}/ E P(1|X.) 
3=1 J ^ i=l ' J 


(9) 


n 


2 = 


p2 = { 2: X P(2|X.)}/ 2 P(2|X.) 

j=l ^ ^ j=l ^ 


The mean y , and covariance matrix, 2 , of the mixture are given by 
m m 


“ “^1 ■*■ ^ a(l-a)(yi-y2) (yj-y2>' 

12 ^ 

Thus from the -gp + + 1 equations in (9), the m.£. estimates of y^ and 2^ 


m m 


are 


12 

given by the set of -|p + |p equations 


n 


i=l 


( 10 ) 


n 


^m = ^ ^ (X-X)(X-X)‘ 

1=1 


Now 


where 


P(1|X) = [1 + ^ + bj 


a = 2 ^(y2-y^) 


-1 


= 1/ T r-1 


T „-l 


b = -|(y^ 2 y^^ - yj 2 y2) + ^n(^) 
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The likelihood funtion can be written in terms of y , E , a, and b, and 

m m — 

1 9 

equations (9) are transformed into the equations of (4) and the 

p + 1 equations 

A T ^ ^ 


which can be written as 


zZ^ (P1-P2) 


m 


a a 


i-a(i-a) (p3^-p2)^2m^(Pi-fl2> 


■'It 1 

b = - - a ^(p^-p 2 ) + 


( 11 ) 


ihe inversion of E^ rather than E might be preferable since E is more apt 
to be ill-conditioned. Now from equations (9), &, and P 2 are functions 
of the X's, ^ and b;- and E^ is given from (10). Equations (11) then form a 
set of equations of the type 

a = <j)^(a,b;x^,...,x^), b = <j>2(a,b;x^,...,x^) 

These equations can be solved by the usual methods; however, the solution may 
be somewhat laborious. The equations, may yield several local maxima for the 
likelihood function. Thus the likelihood function •'should be examined at all 
local maxima so as .to choose the solution yielding the overall maximum. 

According to Day (1969) the m.£. technique in the case of a mixture of 2 
multivariate normals is computationally feasible for p £ 10. 



When the covariance matrices are not equal, the m.l. technique breaks 
down, since each sample point generates a singularity in the likelihood function. 
4. Minimum Chi-Square Estimators 

Let H(x) be a mixture of N p-variate normal distributions F^^, so 


that 


N 


H(x) = ? ct. F.(x), 
1=1 1 1 


( 12 ) 


C- , C».,, C, be k p-dimensional cubes of equal volume containing all n sample 

JL ^ 1C 

points from H(x) , H(C^) be the probability of an observation falling in the 
ith cell and n^ be the frequency of sample points in C^. Then estimates of 
the parameters of the mixture, namely a^^'s and the parameters of the F^^s, can 
be obtained by minimizing chi-square. 


^ - nH(C )r/nH(C )], (13) 

J ^ J J J 

2 

or similar criteria, ' such as modified X , Hellinger distance and Kullback- 
Leibber separator (see Rao, 1965, p 289).. 

In one dimension the mini m u m chi-square estimates can be obtained without 
great difficulty, and as one would expect,. they behave well. In higher dimen- 
sions, however, the computation becomes prohibitive as it involves the evalua- 
tion of p— dimensional normal integrals over a series of cells. 

Hasselblad (1966) used this approach in case of univariate mixtures when 

the data are grouped such that the length h of each class interval is relatively 

2 2 

small compared with the variances a^,.., o^. Assuming h to be 1, he approxi- 
mated F (C ) by the density f. (x.) of F. evaluated at the mid point x. of the 
J ^ J 1 J 

jth cell and obtained his estimates by minimizing given by (13) with 
H(C^) replaced by E a^f^(x^), j=l, ,k. Hasselblad then used his minimum chi- 
square estimates as an approximation to the maximum likelihood estimates. 
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In higher dimension also the computation difficulties Involved in the 
evaluation of ceil probabilities can be minimized by approximating the cell 
probabilities by h^E a^f^(x^), where f^Cx^) denote the value of the density 
function of F. at the center x. of the cell C,, provided the length h of each 
cell is taken to be sufficiently small compared to |e^{ the pth root of the 
determinant of the covariance matrix of F^. However, it can be expected that 
use of such approximate values of cell probabilities in the chi-square criterion 
(13) may introduce serious bias in the estimates. 


5. Least Square Estimator 

A technique, which can be termed "least square" has been proposed by 
(1969) and Choi and Bulgren (1968) for estimating parameters of a mixture 
of univariate distribution functions. The proposed estimator based on a 


rCn>_ 


sample denoted by G(X^^^)=G^=(aY, • . - is 

any G(X^^^) which minimizes Sjj(G) given by 


Choi 

(4) 


Sj^(G) = /[H(x) - H^(x)]^ dH^(x) 


N 

= /[ X a.F(x;0.) 
3=1 ^ 


\Cx)]^ dH^(x) 


1 ^ r ^ i 2 

= n I E X a F(x_,0 ) - 

i=l 3=1 J 3 

where H^(x) is the empirical distribution function of H(x),and X^^j denotes 
the ith order statistic of X^”^ = (Xj^,X 2 , . . . ,X^) . To Illustrate the procedure, 
let m = 1, that is, each is Indexed by a scalar 0^. 
solve the system 


The procedure is to 
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0 


U2(G) = 


Some properties of the estimator (under 

certain assiimptions given in Choi [6]): 

(i) If Gq = ®2’ ' * * ’ denotes the true 

values of the parameters then G^^ ^ Gq with probability one. 

(11) With probability one, there exists a neighborhood of Gq such 

that, for all but finite n, G^ is the unique solution of = 0. 
(iii) G has an asymptotic multivariate normal distribution 

6. Statistically -Equivalent Blocks 

Let X , , X be n observations on a pxl random vector X with distribution 

1 * • n 

function F(x), h^(x), h 2 (x),.,, h^(x) be n functions of x, not necessarily 
different, such that the distribution of h^(x) , i=l,.., n, is a continuous 
distribution function. Then for ‘.all’ i, 

P[h^(X^) = h^(Xg), a 7 ^ e] = 0. 

Also let k^, k^ be a permutation of 1, 2,,., n and X^^l^ be defined as 

that X^ for which hj^ (X^) is the k^ th order statistic among hj^ (X^),.,, 
h^ • Then the cut 

^ h (x) = h (X^^l^) 

1 1 





15 


defines two blocks 


and 


®l“k^ = {x : hj^^Cx) ^ 


B. 


k,+l...n+l ^ 


The procedure is continued. Let 0 < k^ < k^. Then the function h^ (x) is used 

to order k-1 X 's in B and X^^2> ±s defined as that X for which h, (X ) 

a i. . a a 

is the k th order statistic among k-1 Ir (X„) , X.e B,„, . 

^ k^ p B 


Let 


B, 


^r-k^ ®l“kj^ ^ h^^(x^^2^)} 

„+l"k, " ®l"k. VCX^^2>)} 


If k^ < kj, rank the n-k, X ’s in- B according to h, (x) and let X^^20 be 

“ l*‘n+l ^2 

the (k 2 ~k^)th in the ranking. Then 

.(k2). 


B, ..... = B n {x ; h^ (x)^ hj^ (X^*^^0) 


k^+l-k^ 


k^+l"n+l 


and 


+l*n+l “ \ +r‘n+ll^ \ 

y 1 *''0 


At the end of the mth stage there will be m+1 blocks: B , B , ... 

jl jx"^^ ^2 

^jm+l"n+l* Jx***’ ^X’**’ arranged in ascending order. The 

function h, (x) is then used to order X ' s in the block having k . as one of 
m+1 “ 

its indices and x^^+1^ defined to be the X in this block such that k -1 

are either in lower-ranking blocks (blocks with indices less than k 

or ranked lower in this block by block is replaced by its 

intersection. with {x : h, (x) ^ h (X^"^!))} and its intersection with 

m+1 m+1 

^ (X^^^^l^)}. The procedure is continued until after n 

m+1 m+1 

stages there are n+1 blocks B^,.., B^^^. The blocks B^^^^ are called 

statistically equivalent blocks (Tukey , 1947) . The procedure has been 
described by Fraser (1957, section 4.3) and by Wilks (1962, section 8.7). 



Example . To illustrate this method let 



•1 


- ^ -1 

• * 

^1 = 

1 

-2 

.X2 = 

X=l^l X = 

ij, ^2J» % 

2 


1 • 


vT r/i 

- 


1 

’’ ^6 = 

V[t]> ^8 - 

5 

4^ 


be 8 observations. Let (k, , •, k„) = (3, 6,1, 2, 4,5, 7, 8) and let h.(x) = h,(x) = x 

± O O Q 

the first coordinate, and hj^(x) = x^., the second coordinate, for k 3,6. Then 
(3) 

X is the observation vector with the third smallest value of the first co- 
ordinate and = X^. Next, X^^^ is the observation among the latter five 

which has the (6-3) th smallest value of the first coordinate. So, X^^^ ~ ^7* 

Of the two observations, X^^ and X^ having the value of first coordinate less 

(3) 

than the value of first coordinate of X , X^^ has the smallest second coordi- 

nate and X^ the second smallest. So, X^^^ = X^^ and X^^^ = X^. The process is 
continued until is obtained. The blocks B.., '..., B- are shown in 

JL O 

the following figure. 







«3 . 


(6) 

X 

i 


i 

„(2) 

“5 

x<« 




^<7) 







®4 



Extension of Least Square Method 

Using the concept of statistically equivalent blocks, the least square 
method of Choi can now be extended very easily. Now we have. 
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where 


r*f i 

F(X''"-^ 6 .) = S F.(B,), F, = F(x, 0 .). 
J k=i 3 k J :i 


However, the evaluation as F,(B ), the prohability content -of B under F.(x), 

j k K j ■ 

may be very difficult. 

7. Comparison of Estimation Procedures 

In sections 2-6 we have discussed some of the most interesting procedures 
for estimating the mixing distributions of mixtures of univariate and multi- 
variate identifiable distributions in the absence of training samples from the 
component distributions. The performances of these estimators are usually com- 
pared on the basis of bias and mean square error. But the sampling distribu- 
tions of these estimators are extremely difficult, rather impossible, to obtain 
analytically. Robertson and Fryer (1970)' obtained some approximate expression 

for the bias of moment estimators of a special type. Hasselblad (1966) was able 
to obtain graphs of variances of the estimates of the proportions (mixing dis- 
tributions) against population distances in case of mixtures of univariate 
normal distributions. On the basis of Monte Carlo studies Day (1969) has 
tabulated the mean and variances of the moment estimators and m.l. estimators 


of the mixing distribution (a, 1 -a) of a mixture of two normal distributions 
with equal variance or covariance matrix. Choi and Bulgren (1968) have also 
tabulated the -mean square error of the least square estimates of the mixing 
distribution of mixtures of univariate normal distribution. 



II Training Sample Available 


In the following section we discuss some procedures for estimating the 
mixing distribution G(a^, of mixtures H(x) , given by (4), when we 

have training samples from the component distributions For the sake 

of simplicity we shall restrict our attention to mixtures of two populations. 
Throughout our discussions in the following sections we shall assume that 

^ sample from the first population H_ , Y , , Y a sample from 

. 1 1 • n2 ■ 

^2 aiid ^l»*’^n ^ sample from E, a mixture of .and E2. The distribution 
H(x) of the mixture is given by 

H(x) = a^F^(x-) + “l **" 

8. Confusion Matrix Estimators 

A detailed account of this estimation method can be found in the papers by 
Odell and Chhikara ( 1974 ). In this procedure unlabelled sample from E is 
classified by an arbitrarily preassigned classifier C. Let e^ and 62 be the 
expected proportion of the sample classified under E, and ^2, respectively 
and as a result of using the classifier let (|)j^ = P(E^{E2) and - P(E2.|e^) be 
the respective probabilities of misclassifying an element from E2 into E^ and 
vice versa. Then the true proportions of E^^ and a2(=l~ctj^) of E'2 are given 
by 


e^ = (l-i})2)a^ + (|>^.(l-a^), 62 = 1-e^ 


( 16 ) 


The matrix 



•^1 


has been called confusion matrix . The misclassification probabilities are 
estimated from the training, samples and the expected proportions from the 
unlabelled sample. There we have 
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a - (e^ - ( 17 ) 

if cj)^ and ^2 known and 

a - - c|)^) / (l-i}»^-(J)2) . (18) 

if and are unknown, ^ denoting the estimate of the quantity 0. 

Var a = Var (e^) / (i-<j)^-(j)2) ^ (I9) 

Some approximate expressxon or the mean square error or a can be round in 
}< VA.J'7 // {'/ 

Odell and fc hhil t a - ra ( 1974 )? ^ 


9 . Least Square' Estimators 

Let the component distributions and F2. of the mixture H(x) be known. 
Then in case of univariate populations, minimizing 


S (G) - E [ E a F (Z. .) - i/n] , 

^ i=l ^ 1 (1) 

where Z.,. is the i-th otder. statistic of Z-, Z , we obtain 
(.1) 1 n 


n 


cc, = 
1 




L - ^2«(i))l" 


( 20 ) 


and 


Var (a^) = E^^ |f^(Z^) - F2(Z.) i ^H(Z^ [l-H(Z^) ] 


” ' 1-1 


( 21 ) 
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In the multivariate case, one way to obtain an estimate of a, and an 
expression for its variance may be to replace j=l,2, in (20) and 

(21) by Fj (B^) , where are the statistically equivalent blocks 

determined by any prechoosen ordering functions h^(x), h^(x) and the sample 

^1’ •’ Since evaluation of (B^^) may be extremely difficult, the least 

square method may not work very well in case of multivariate data. 

In crop acreage estimation problem we are interested in estimating the 

mixing distribution only. So, the following estimation procedure may be of 

interest. Let F .(x.), F „(x^), . , F . (x ) be the marginals of the distribu- 
j-i- -1- ^ JP P 

tion function F , (x) , j==l,2 and H^(x^), ,, H (x ) that of H(x) . Then we have 

j X X p p 

“l^ll^^l^ ^ (l-a^)F2^(x^) = H^(x^) 

“l^l2^^2^ + (l-a^)F22(x2) = 




An estimate of can be obtained by minimizing 


■ L Cl ' Cl 


( 22 ) 


where is the i th order statistic among . , Z^^, the k th components 

of Z , , Z . Thus we obtain 

1 • n 

“l " ^=1 ^i=l ^^lk^\(i)^ ” ^2k^\(i)^^^^2^\(i)) ■ 

2»P ^ T-» \ 1 2 


and 


^=1 "Li 

var (Sp . 2”.^ I F^^(Z.p - [l-H(Z.p] 


( 24 ) 


“k=l I ^lk<hk> - ^2k<"ik>i'>' 
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When F^(x)'s are not completely known, but are only known to belong to 
certain parametric family of distribution functions and we have training 
samples from both F^(x) and ^ fairly good estimate of and F 2 can be 

obtained by plugging in the estimates of their parameters in their functional 
forms. An estimate of can now be obtained from (2o) or (23) by replacing 
F^(x)'s or their estimates. The sampling distribution of is 

extremely difficult to obtain analytically. So, we have been unable to give 
any expression for bias and mean square error of a. 

10. Moment Estimator 

Let the p-variate population II be a mixture of N p-variate populations 

n, , . , n such that 
X m 


H(x) = a^F^(x) + ••• + = 1, (25) 


where H(x) is the distribution function of II and F. (x) that of II. and a 's are 

X XX 


proportions of in II. Then 


N 

/g(x) dH(x) = fgCx) dF^(x), (26) 

where g(x) is any monotone function of x such that all the integrals involved 
in (26) exist. When a, 's are unknown, but H(x) and P.(x)’s are known, then 
a^’s can be estimated from the following set of N=1 equations 


N 

/gj^(x) dH(x) = dF^(x), i=l,“,N-l, (27) 

where gj^(x),,., g^_^(x) are (N-1) linearly independent monotone functions such 
that integrals on both sides of (27) exist. When F^(x)’s and/pr H(x) are also 
unknown, it has been proposed by Hartley (1974) that for estimating the a^'s 
the moments of gj^(x)*s in the system of equation (27) should be replaced by 
the corresponding sample moment, provided one has N sample from F^(x)’s and a 
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sample from H(x) . Let T",.,, Y"'' be a sample from the p-variate population H(x) 

and X^^ a sample from the p-variate population F^(x), i=lj.., N; also 

let U- , V„ be mxl vectors defined as follows. 

1 1 N 

“3 ° ’'j ■ ^=1^3/” '’13 ■ =^^3 ''”1’ 1 ^ 3 P" 


.k „k 


i ,.k „k 


"j+P " \=1 ^1 


i,j+p k=l “ii “ij 


X., /n., i=l, , N; 1 j ^ 
ij i’ 


j+2p = yJ Y^ /n, V,. , 


„k „k 


k=l 2 


j ^i,j+2p = \=1 \2 ""ij N; 2 O P 


“3«P+(P-D - Vi ’'a ^i.3«p-Hp-i) - ‘^‘i=i^i3^3/"i’"“"- ^ ^ ^ 


Bi „k „k 


% - »rm=P+Pi^ 


Then estimates of ct^'s can be obtained as a set of values 0^ for which 

Q = 1 I U - ? . V . I 1 


is a minimum, | | • j | denoting some suitable norm. Apprehending difficulty in 
minimizing Q based on l2~norm. Hartley (1974) has suggested that 1^ - or 
weighted 1^~ norm should be used as the norm. Thus, when 1^- norm is used, 

m N - • , 

Q = Z. T u, - Z. ^a. V.-, 

2=1 ' 3 x=l 1 13. ' 


or when weighted 1^- norm is used. 


m N , 

Q = ^3 I "3 ■ ^i=l “i ''ij ' 


where are proportional to the inverse of sample standard derivation of u^« 

We shall refer to these estimates as moment estimates with minimum weighted l -j -norm 
deviation . As can be expected, these estimates are very sensitive to the choice 

of weights. 
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Feiveson (1974) used this method to estimate proportions of 9 crop 
classes. He found appreciable change in the estimates due to 
(i) change in weights beyond 3rd decimal place, 

(ii) translation of the entire data. 

The estimates are not expected to be translation invariant. It is believed, 

that the relative sample sizes n-/n,.., n /n will also influence the estimates. 

1 m 

This method of estimation has also been found to do better when the number of 
classes are smaller (say 2 or 3) . 
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Zl 

M EMPIRICAL COMPARISON OF FIVE PROPORTION F<?TiMATnR<^ 

W, A. Coberly^ and P. L. Odell^ 

XntFoductlon; Let irpo..j tT|^ be m pattern classes having 
probability density functions respectively. Let a = 

C®^p..cj c^} be the proportion vector defining the mixture density 

m 

0 ) f(x) = r «. f.(x), 

k=l ^ ^ 

where ^ k=ljo.o, m. The measurement vector x is 

n - dimensional. Consider the following problem: Given a random sample 
Xp,.,, Xj^ from the mixture distribution, find an estimate for a. 

The density functions fp,,,, f|^ are assumed to be known. A short 
discussion of each of the five estimators considered in this paper 
follows: 

Classification Estimator (CL) Define 

'l if f,^(x) > f .(x) jlk 
0 otherwise 

for k=l,.,„, m. Now the CL estimate is given by 

r ^ A 1 N 

oj, - ij- Xj^CX^ ) k=l,,..,m. 

A 

Simply, aj^ is the proportion of the sample whicn wouio oe classified 
into class by the maximum - likelihood classifier. 


^University of Tulsa 
2 

University of Texas at Dallas 
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O dell -Chhikara Estimator (OQ Let P be the confusion matrix 
defined by the maximum - likelihood classifier. That is, P = (p..; 
where 


Pij = 




■= 1 M ^ ITj ) 


for 1, 0=1,,.., m. Simply, p_ is the probability of classifying an 
observation from wj into v,. If, £2j it is shown the ECa] = Pa, 
where a is the CL estimator defined in (2). This relation suggests 

an estimate of „ given by the least squares solution of the matrix 
equation 


(3) 


A 

a = Pa 


:onstrained by Za. = 1 and a. > 0. Denote this estimate by g. The 
properties of g are discussed at length in [1]. 

Ma ximum - Likelihood Estimator (ML). For the sampl e , , . . , 

:he log - likelihood function with the added constraint Za. =■ l gives 
he objective function 


m 


I'M = log ^ f(x.) - X (z a.-l) 

i=i ' k=i I* 

N 


m 


= if, ’“5 ,f,“kfk<Xi) - 

3W dL/da = 0 implies that 


-')■ 


1 " m 

■ N .f, 


( 4 ) 
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tor m. Let G ■ be a vector valued function whose components ‘ 

are defined by the RHS of (4). Then the ML estimate of a, if it exists, 
must be a solution to the fixed point equation 

a - G(«). 

The numerical behavior of this optimization problem Is discussed further 
in 13]. 

Estimate (MX) The equation relating the mixture distribu- 
tion function and the component distribution functions corresponding to 
(1) Is 

m 

F(x) = E «kFJx). 
k=l 


Denote the corresponding jth marginal dist 
C3 ) 

and Fj^ respectively, j=l,,,., n. Consider the following system of 
equations. 


( 6 ) 


:(j) 


m 


j“l,..., n, 1-1,,.., s. In this experiment the functions were 

estimated from the sample Xj^ and x^.^ was chosen to be the 

1001/{S+1) percentile of Define a vector Y and a matrix A by 


y =F*^^{x..) 
'^p ' 13^ 
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for p (j“l)s+i, n, s. Then the mixture estimate 

MX is defined to be the least squares solutions of the linear -system 

(7) Y = Aa 


constrained by Za^. - 1 , a^. > 0. (It is assumed that for each j the 
percentiles x^.^. i=l s are unique. If not, the redundant equa- 

tion; is deleted.) 

Moment Estimator (MO) Let n and denote the jth com- 

ponent of the mean vector for the mixture density and the kth compo- 
nent density k=l,..., m. Likewise let y.. and y^. denote the ijth 
element of the noncentral dispersion matrix of the mixture density and 
the kth component density. From (1) 


for i=o,...., 
sample X-j,.. 
and a matri 


n, n. Estimate the raw moments y.. from the 

^ \J 

• » Xfij and denote these by Pjj* Now define a vector Y 

* *J 


X A by 


^oj’ ®jk " 


for j=l ,. . . , n and 


^p ~ ®pk 


where p = n+(i+l)i/2+j 
Then the moment estimate 
tion of the linear system 


and 

MO 


i 1 ,..., n; j~l,..., i and k = 1 ,.,., m. 
is defined to be the least squares solu- 
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(8) WY - WAa 

constrained by = 1 and > o. Here W is a diagonal weighting 
matrix defined in" this experiment by 

~ I ■ 

1 for p~ n^" (i'^l)i / 2 ^" j 3 i~ls ...3 ns j^ls***# i* 

A comment is now in order about the assumption that the component 
densities fj^ are known. In this experiment the densities are esti- 
mated by sampling labeled or ground truth data prior to the estimation 

of the proportion vector a based on the mixture sample Xj^. 

Hence the density estimates and the proportion estimates are found in- 
dependently. This distinguishes the problem posed here from the 
general mixture problem in which the component density functions and 
the proportions are estimated simultaneously from the- mixture sample. 
That is, no labeled data is assumed available. 

II. Experimental Procedure The data base used for this experiment 
consisted of ERTS-A 4-channel multispectral scanner data, taken on 
May 5, May 23, June 11, and June 29, 1973, over a 14 square mile test 
site in Hill County (N) ,. Montana. Only the data from the June 11 pass 
was used in these results. A ground truth map and summary dated May 5, 
were used in conjunction with the Earth Resources Interactive Processing 



System (ERIPS) at NASA/JSC to identify and tag pixels, which fell into 
recognized homogeneous fields of one of five general classes: Wheat, 
fallow, barley, grass and stubble. Of a total of 8400 pixels in the 
test site, 2600 were labeled accordingly. The training and test samples 
were determined as follows. A random sample of 30% of the labeled data 
was selected for the training data set and (after replacement) another 
30% random sample was selected for the test data set. For the training 
set, each pixel selected was grouped according to class tag and statis- 
tics and marginal histograms for the total 16 channel data set were 
computed and saved for each class. The test data was then classified 
using the training statistics (June 11 pass only) and the confusion 
matrix estimate was based on those results. This procedure was repeated 
30 times. The five proportion estimates described in the previous sec- 
tion were then computed for each of the 30 trials, first using the 
2600 labeled pixels as the mixture sample (Experiment I) and second 
using the total 8400 pixels (Experiment II). The results of each of the 
30 trials for both experiments are exhibited in Appendices A and B. 

Five major computer programs were required. CLASS computes the 
CL and ML estimates. This program reads each set of training statis- 
tics, classifies the mixture sample and computes the. CL estimate. Fur- 
thermore, as the values of the density functions f-,(X^ ),..., f^(X.j) (m=s) 
are computed to make the classification decision, they are written 
to temporary storage for use in computing the ML estimate, A simple 
iteration method was used to solve this fixed point equation a = G(a). 



The initial guess, denoted by a° , was taken tobd CL estimte 
computed, then successive approximations were found by the following 
rule. 

= G(a'^ ^), n=l , 2,... 

until li II was "sufficiently small." In this experiment 

ten iterations were sufficient to achieve 3 or 4 place convergence. 

The CONF program reads the training statistics and classifies a random 
sample of the labeled data in order to estimate the confusion matrix. 

The ODE program reads the CL estimates and the confusion matrix esti- 
mates and finds the constrained least squares solution of (3), The 
MIXTUR program reads the mixture and component histograms, sets up the 
linear system given in (6) and finds the constrained least squares solu- 
tion. The MOMENT program reads the mixture and component statistics, 
converts them to the noncentral moments, sets up the linear system 
given in (B) and finds the constrained least square solution. The last 
three programs require subroutines PREP and QUADPR which set up the 
quadratic objective function and solve the resulting quadratic program- 
ming problemjrespectively. The program QUADPR was adapted from [2], 

All programs were coded in Fortran and are listed in Appendix C. 





Table 1. Summary of Experiment I. 

{Labeled Data, 2600 Pixels) 


CLASS ODELL MLE MIX MCH GT 

*****************************************************************************************^*^****^^^^*^ 


MEAN 

WH 

.282384 

.297041 

.300439 

. 294353 

.274 49 

.371900 


FA 

.265500 

.274428 

. 296764 

.312696 

.235257 

.286200 


BA 

.186089 

.174306' 

.176688 

.183300 

.197059 

.115400 


GR 

.101679 

. 086066 

.085825 

.074200 

.075168 

.079200 


ST 

.164346 

.168155 

.140178 

.135448 

.217765 

.147300 

VAR 

WH 

.000038 

.000174 

.000083 

.000201 

.000459 



FA 

.000445 

.001385 

.000408 

.002216 

.003027 



BA 

.000031 

.000190 

.000084 

.000123 

.000288 



GR 

.000080 

.000051 

.000060 

.000165 

.000463 



ST 

.000344 

.001829 

.000383 

.002376 

.004145 


TOTAL VAR 


.000937 

.003628 

.001017 

.005020 

.008383 


MSE 


.015172 

.013322 

.010086 

.016572 

.032066 




Table 2. Summary of Experiment II 

(Total Data Set, 8400 Pixels) 

CLASS . ODELL MLE MIX MCM GT 

*************************************ie*i(*ic***-Ki^^7^^7^T,^*i(iei(***********i!ic4t*****i(**i(i(**ic*i:it*****i(**-k-k*-k-k** 


MEAN 

WH 

.252733 

.262044 


FA 

.207300 

.183526 


BA- 

.167633 

.154997 


GR 

.186466 

.187198 


ST 

.186066 

.212231 

VAR 

WH 

.000095 

.000263 


FA 

.000269 

.007430 


BA 

.000063 

.000299 


GR 

.000261 

.000363 


ST 

.000141 

.009569 

TOTAL VAR 


.000829 

.017924 

MSE 


.008622 

.057982 


.272700 

.226467 

.084333 

. 294000 

.185133 

.353019 

.023415 

. 249000 

.151200 

,218460 

.325809 

.124000 

.189166, 

.158955 

.184282 

.138000 

.201700 

.043098 

.382159 

,196000 

.000310 

.000232 

.000949 


.000472 

.002872 

.002114 


.000206 

.000135 

.000610 


.000529 

.001042 

.000900 


.000832 

.002975 

.003402 


.002350 

.007257 

.007974 


.010273 

.055378 

.180347 




III. Results and Conclusions In Experiment I only the labeled subset of 
the Hill County data was used. In Table 1. the mean and variance of the 
components of the proportion vector estimates are tabulated, averaged 
over the 30 trials. TOTAL VAR is the sum of the variances and MSE is the 
mean square error about the true proportion vector given in the column GT. 
It should be noted that one wheat field, accounting for 6.4% of the labeled 
data was consistently described as barley, apparently- by all five estima- 
tors. Hence the discrepancy between the ground truth and the five esti- 
mates with respect to wheat and barley. 

If however, the 6.4% is added to the Barley Class, then 'the MSE in 
Table 1 . would read MSE .002858 .004393 .001286 

.018915 .006146 which reverses the ordering of (MO, MX) and (CL, OD). 

The ML estimator remains the .best. Otherwise all five estimates apparently 
will be relatively stable with CL and ML having the lowest variance and 
MSE with MO apparently having a significantly larger variance and MSE than 
the others. A trial by trial summary is given in Appendix A‘. 

■ In Experiment II the total Hill County data set was used. In Table 2 
the GT column is not precise since- many small classes listed in the ground 
truth summary were arbitrarily consolidated into the five class model. 
Conclusions drawn from the MSE should reflect this uncertainty. The CL 
and ML estimates are again lowest in variance and MSE followed by the OC 
and MX estimates. However, in this experiment the MO estimate is very 
bad. Apparently the moments of the total data set deviated sufficiently 
from the five class model to cause the distorted results. In Experiment I 
when only the labeled data was used the model more accurately reflected 
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the sample and the MO estimate performed reasonably well. If further 
studies find this lack of robustness to be a consistant problem, then the 
moment estimate should be eliminated from consideration in the applica- 
tion. 

Based upon the results of these two experiments the following order- 
ing of the five estimates is suggested: 

(CL, ML) > (OC, MX), > MO 

Several operational considerations should also be noted however. 

The MX and HO require design decisions prior to implementations such as 
choice of the percentiles and weights used in the construction of the 
linear systems in {, 6 ) and C8) respectively. In an automated system thes= 
additional parameters might be considered as a nuisance. The OC estima- 
tor requires allocation of some labeled data in order to estimate the 
confusion matrix. An alternative would be to use Monte Carlo-methods on 
the model described in [ 1 ] to obtain numerical approximations of the con- 
fusion matrix given the estimates of the component densities. The CL, 

OC and ML estimators require classification of the total data set and 
hence require much more computational time than the MX and MO estimates. 

In conclusion, on the basis of this study, the maximum likelihood 
estimator would appear to be the best of the five with respect to MSE, 
numerical stability, and adaptability in an .automated system. 
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I. INTRODUCTION 

Consider m different populations tt, , ir-* .... it and a random 

1 z m 

sample of n values from these populations. Let n. be the number of 

1 

points classified into i = 1, . . . , m, using a classification al- 
gorithm. If n (i[j) is the number of data points classified into 
but actually belonging to then 


^i ~ ■*' + ... + n(i[m) 


Also, 


n. 



m 

2 

i=i 


n 


i = 1, 2, .. 


m 


( 1 . 1 ) 


are the observed proportions for the sample data under the class- 
ification algorithm used. The observed proportion n./n is a biased 

1 

estimate of p^, where p^ = the actual proportion of the sample that is 
in TT. . n /n is an unbiased estimate of e., where 

XX i’ 



■ L ■ 


= 2 p . P(i[j) 
J=1 ^ 


( 1 . 2 ) 
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and where P(i|j) denotes the probability of classifying a data point 
from TTj into under the classification algorithm. 


If P(i|j) = 0 for i j or n(i|j) = n(j[i) for all i,j then 
~ Pi i ~ 1» 2, • • • , m, i,e. the sample proportions provide un- 
biased estimates of the actual proportions. In general, a classifica- 
tion algorithm will be subject to error and these two proportions 
will not be the same. 


Denoting the - observed proportion i, by 6 , then by (1.2) it fol- 

'n ^ 

lows that 

e = E(g) 

== Pp (1.3) 

where 




P(1 

1) P(1 

2) ... P(1 

m) 

and P = 

P(2 

1) P(2 

2) ... P(2 

m) 


P(m]l) P(m|2) ... P(ra|ra)J 


Considering the case where P is unknown, th 


Pp 


= e 


(1.4) 
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and an estimate of ^ is 

? = e (1.5) 

where P is an estimate of P , P non-singular. 

A, 

P is clearly a biased estimate of p, where both bias and mean 
square error of p depend on the performance of the classification al- 
gorithm as well as the degree to which the sample represents the pop- 
ulation. 

Consider the case for m = 2. If N sample values are used to 
estimate P(l|l), P(2[l) and N sample values are used to estimate 
P(l|2), P(2j2) then 



where 

X = the number correctly classified into iTj|^ 
y = the number correctly classified into it 2 . 

Av 

Now, P is singular iff x+y = N. Since X and Y are Independently 

A 

distributed as binomial variables, then P is singular with positive 

A 

probability. Consequently, the pseudoinverse of P must be employed 


for these situations. 
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Definition 1.1 A matrix A is called a pseudoinverse of A if 

it satisfies the following conditions 

(i) AA'^A = A 
(li) A+AA+ = A-^ 

(iii) (AA"*")^ = AA^ 

(Iv) (A^A)^ = a"^A. 

Theorem 1.1 Each matrix has one and only one pseudo- 

inverse. [1 ] . 


II. DEFINITION OF PROBLEM 


The problem is to 

• . • .A ^ 

Min I ] Pp - e j | 
T 

subject to J p = 1 

p 0 


where = (1, 1, 1, . . . , 1) .* 


( 2 . 1 ) 


(2.1) is equivalent to solving 

Max (e Pp - ^ pP Pp) 

T 

subject to J p = 1 (2.2). 

p > 0 


Another approach would be to solve 



subject to p ^ 0. 

This approach may not produce a solution vector p , which sums to 
(exactly). For this reason (2.1) will be solved Instead. 



Two methods for solving (2.2) will be discussed. The first method 
(Iterative Search Method) is due to D. L. Nelson [3 ] , and the second 
method uses a modified version of the Simplex procedure. 


III. Iterative Search Method (Method I) 

Define C = (I - - JJ*^) (P^ P) (I - ^ JJ^) 
m m 

and H = (I - - JJ-^) P^ (e - - P J) 
m ' ^ m 

Let p* = i J + P^ (e - ~ P J ) 
y m m ' 

1 T + 

+ (I - - JJ - C C) y 
where y is any mxl (real) vector. 


(3.1) 


Theorem 3.1 


If p* > 0 for some y, say yo, then p* is an • 
^ “ yo 


optimal solution to (2.2), If p*^ <0 for all vectors (i.e. p*^ > 0 
is infeasible), then let { p , i = 1 , 2 , ..., k} be' the negative 


n. 

X 


components of P*y* Then there exists a vector p which satisfies 

(2.2) such that at least one of the p =0. 

n. 


To determine if p^* > o has a feasible solution use the Phase I 
procedure of the Simplex method. This procedure will determine if 
there is a feasible solution to the matrix inequality 
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A y < b 


(3.2) 


where A = c'*'g + — jj'^ - l 

m - 


b = - j + p-^ (e _ i p j) 

m m 


Using any linear programming routine, solve 


m 




Min (J) = E (v. + V. ) 

^ ^ r 1 


subject to (A -A I I -I) 



= b 


where S = slack variables 



V = 




Consequently, (3.2) has a solution iff the sum of the artificial 
ariables equal zero, i.e. i|) = 0. If this is the case, the y vector 
atisfying (3.2) is given by 

+ — 

y = y - y 
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For the case where no feasible exists, then by Theorem 3.1, 
a solution to (2.2) exists of the form (3.1), where at least one of 
the components is equal to zero. For such a situation, two cases are 
considered. 


Case A: CH = H 

Step 1 

Choose any real vector y, say y . 


Let S={i|p* >0} 

^oi- 

and T = { i | p* < 0 } if‘ <i> , 

^ol 


* ^ 
where p = the i-th component of p 


For each i e , set p^ — 0 and see if a i.uasj.oj.e soiucxon 
exists in the (m-1) - dimensional space which remains. This is 
equivalent to removing the i— th column of P and solving for p as 
before. 

bet = {i I i£T^ and p^ = 0 provided a feasible solution} 
and T^ = {i | iST^ and p^ = 0 provided an infeasible solution}. 



47 


If go to Step 4. Otherwise continue to Step 2. 

Step 2 

Consider all pairs of components (i» j) such that i eT^ 
j 4 ^1 ~ remaining (m-2) - 

dimensional space, if possible. This is equivalent to removing 

■s ^ 

the • i-th and j~th columns of P and again solving for p as 

y 

before. 

Let = { (i, j) ( = Pj = 0 provided a feasible solution} 

atid ^2 ~ { (is j) I ~ Pj = 0 provided an infeasible solution}. 

If = <j), go to Step 4. Otherwise continue to Step 3. 

Step 3 

Do the same for all triples (i, j , k) such that at least one 
element G and such that (1, j), (±, k), (J, k) ^ S 2 . 

Next consider 4- tuples, etc,, until all tuples have been con- 
sidered or ^ for some 1, , 

Note : In Steps 2, 3, at no time should a set of components be set 

equal to zero when some proper subset (set equal to zero) brought a 
solution. For such a situation, the objective junction in (2.2) will 
not be improved [ 3 ] . 
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Step 4 

After a finite number of steps, the optimum solution can be 
derived by determining which vector obtained in Steps 2, 3 maximizes 
the expression in (2.2). 


Case B: CH H 


Step 1 ' 

Attempt to find a solution by setting 
= 1, 2, ..., m, one at a time. 



for j 


Let S^ = {i I p^ = 0 provided a feasible solution} 
and = {i | p^ = 0 provided an infeasible solution} 


If = <|), go to Step 4''. Otherwise go to Step 2'*. 

Steps 2', 3"', h' are exactly the same as Steps 2, 3, 4 
(respectively) for Case A. 

IV. Simplex Method (Method II) 

Let f = e P and D = P P. 

Thus (2.2) can be written 

1 T 

Max p = fp- — p Dp 
o 2 

T 

subject to J p = 1, p ^ 0. 
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Let g(p) = J p - 1. 


Imposing the Kuhn-Tucker conditions for this problem (see 
[ 2 ]), there must exist a set of p^ and X such that, for each 


9Po ^ 9g.(j£l 


9P^ 




0 j = 1 


9 • • ♦ J 


m 


(4.1 


X , unrestricted in sigr 


Thus (since D is S3nnmetric) 


f - Dp + J < 0. 


Introduce a set of slack variables S = (s, , s„, s )' 

12 m 


such that 


f -Dp + Xj + S«0' . 


(4.2) 


As a consequence of (4.2) (see [2 ]), it follows that p^s^ 
must equal zero in the optimum solution. 


A starting feasible solution is p^^ = 1, p£ ~ P 3 ~ • • * “ ” 

0, and . so Phase I of the Simplex method can be omitted. The second 
step is to Introduce a set of non-negative artificial variables 
Vj . These are subtracted from the equation set (4.2) and the 
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objective function is defined as the sum of these artificial variables. 


Letting ? = v^, . . . , 


T 

, then the problem is 


Max -J^ V 


(4.3) 

subject to (a) = 1 


(b) - Dp + j + S - V = 0 

(c) p^s^ = 0 

(d) p > 0 . 


(4.3) can be expressed in matrix notation as 


Max V 


(4.4) 


subject to 


B - J J - I I 

0 0 0 0 






2 : 0 , X2 > 0 , p > 0, s > 0, V > 0, 


First, (4.4) will be put in canonical form with starting babj.s 




V . 

m 



The constraints in (4.4) can be written 


d^^ d^2 ^i^m ^ -1 0 . . .0' 1 0 . . .0 


"^i 

*^21 ^22 *^2m ^ ® ^ 

• • • • « « • 

• • # ^ ♦ • 

• • * * « 

= 

^2 

d. d „ d -1 1 0 0 . . .-1 0 0 . . .1 

mi mZ mm • 


^m 

1 ... 1 0 0 00 000. ..0 


1 

L.'ii 


The first step is to remove d,-, d Thus row (m + 1) is 

i-L mi 

multiplied by — d.- and added to row i providing 


— — 


— 

0 d^2 "• 1 -1 0 ...0 1 -0 ...0 



0 d" dr -1 1 0-1 0 0 1 ...0 

22 


^2 

• 

* • *•••««•« 

• • •••••••• 


• 

• 

0 d"_ d' -1 1 0 0 ...-1 0 0 ...1 

m2 nn 


f' 

m 

1 1 ... 1 0 0 0 0 ...0 0 0 ...0_ 


1 


where d'T . = d . ^ - d . , 

13 ij il 



The next step is to multiply each row by -1 for which 0 , 

except for column (2m + 2 + i) . This remains a positive 1 . For sake 
of discussion, it is assumed that f^ ^ 0 for i = 1, . . . , m. Writing 



the objective function as the first row, the simplex tableau becomes 
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where 0^ ■= “ i = 2, m 

3=1 

m 

1=1 

Note that the first row may differ slightly if previously any 

of the rows were multiplied by -1. Colinnns 1, 2m + 3, 3m + 2 

will always contain zero. The simplex method can now be applied to the 

above tableau with the restriction that s . and p , cannot be in the basis 

3 3 

at the same time. This algorithm can be easily coded on a digital com- 
puter. 


V. Examples 

A 

A. m = 3 , rank ( P ) = 3 


P = 



.3 

.6 

.1 






(i) Method I 


P 


A 

y 



(0)y 


(Infeasible) . 


The optimal solution is found for P 2 = ^» where 


P* 

y 



(0)y , i.e. p^ 


.605, p^ 


.395. 



(ii) Simplex Method 


The optimal solution is given by 


= . 605 

P3 = .395 

^2 = .018 

= .040 


All other variables = 0. 


Here it can be seen that Nelson's procedure is rather inefficient 
since it was necessary to examine all solutions for which p^ = 0. For 
each method, the norm in (2.1) = .0056 > 0. 


m = 6 

, rank 

( P ) 

= 6 







".71 

.04 

.13 

.01 

.03 

.05 


.10 

P = 

.02 

.82 

.06 

.11 

.02 

.01 

' e = 

.18 


.11 

.01 

.66 

.03 

.07 

.02 


.22 


.03 

.05 

.04 

.75 

.01 

.01 


.28 


.06 

.03 

.08 

.06 

.85 

.03 


.12 


.07 

.05 

.03 

.04 

.02 

.88 


.10 







— 




(i) Method I 


P* 

y 


.065 

.148 

.294 

.343 

.077 

.073 


+ (0)y 


Since each P^ >■ 0» then the above is an optimal solution. 
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(ii) axmplex method 

The optimal solution is 

p^ = .065 

?2 = -148 

P3 = .294 

= -.343 
P 3 = .077 

Pg = .073 

= 0.0 

All other variables = 0. 

The norm in (2.1) in each case = 0.0 

A 

C. m = 6 , rank ( P ) = 4 


71 

.246 

.13 

.040 

.03 

.05 


.10 

02 

.052 

.06 

.015- 

.02 

.01 


.18 

11 

.550 

.66 

.045 

.07 

.02 

/N 

.22 

03 

.038 

.04 

.010 

.01 

.01 


.28 

06 

.076 

.08 

.440 

.85 

.03 


.12 

07 

.038 

.03 

.450 

.02 

.88 . 

■A 


.10 


(i) Method I 


'*.1248'" .024 -.119 .095 0 0 C 

* .2410 -.119 .595 -.476 000 

p = .2701 + .095 -.476 .381 0 0 0 y 

^ .1214 000 .667 -.333 -.333 

.1233 000 -.333 .167 .167 

.1194 000 -.333 .167 -167 
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Thus for any y, the corfesponding is an optimal 
solution to (2.2). 

(ii) Simplex Method 

The optimal solution is given by 

/V 

p^ = .173 

A. 

p^ =" • 463 

= . 360 

= . 004 

= .084 

= 0 
o 

'' 19 ° “ 

All other variables = 0. 


Using a Phase I procedure, it can be shown that for 



the result using Method I reduces to the above results 
using the Simplex method. 

Using either method, the value of the norm in (2.1) 


is .1242. 



VI, Mean Square Error of p 

y 

First consider the possible values for P. If n, sample values 'are 
used to estimate the first column, n^ to estimate the second column, etc. , 
then 


n(l|l) . 
n. 


n(l m) 


n 


m 


P = 


n(mll) 


n(m m) 


n. 


n 


m 


( 6 . 1 ) 


Where n(i|j) is the number of data elements from population tt. 
that are classified into population ir^. Assuming that the m samples 
(one for each column) are independently taken, then column j has a 
multinomial distribution with probability 


5^(1 1 j) I n(mjj)! 


n(l[j) n(2|j; 




n(mjj) 

P . 
mj 


Where p^^ = (i, j) -th element of P 
= P(i|j) 


Theorem 5.1 

/V 

If P is defined as in (6.1), 

A 

possible values of P. 


m 

then there are Z = ir 

i=l 




Proof: The proof will be constructed using a generating function. 

Consider the expression 

9 n 

(1+X+X +...+X )™ 


( 6 . 2 ) 
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Consider (6.2) as the product of m terms of the form 
(1+x+x +...+x^). Consequently the coefficient of x ^ in the 
expanded expression In (6.2) will be the total number of ways of con- 

✓S 

structing the i-th comumn of P. This is generally called the number of 
ordered partitions of n^. Now, 


n. 


(1 + X + x'" +. . . . + X ^)® = 


n,+l 

(1-x ^ )”^ 

(l-x^“ 


Also, 


V ^m— l"f"iv 1 

,m - ? „ ( 1 ) X , 


(l-x)“ 1=0 


X < 


(6.3) 


Thus the right hand side of (6,3) can be written 

n.+l oo 
(1-* )" Z 

i=0 ^ 

^ • 

The coefficient of x in (6.4) is clearly 


z. 

X 



Thus the total number of possible values for P is 


(6.4) 



By the previous discussion, for each value of the matrix P, say 
(^53.12), there is a corresponding probability of obtaining this P., 


say q . . Thus 
J 
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m 


n = TT V 

^ ^1 -1 P9-1 * * * P • 

3 1 J-i Zx *^mx 


where k . - 


n,! 

X 


"■ n(lji)I 


n(m|i)I 


= n(l|x) + n(2[i) + ... -, 

^ Pli ^21 “^ • • • ■^ •• 


mx • 


If P * given by (3.1) is ^0 for some y, say y , then for 

^ O 

some y — y^ (possibly y ), p* ^ 0 (hence optimal by Nelson’s procedure) 

■ ° ^1 

and is also equal to the solution obtained by using the Simplex Method. 

If there is no y such that p* ^0, then by the previous discussion, 
there does exist a solution which minimizes the norm in (2.1) such that 
at least one of the components equals zero. Setting one of the components, 
say Pj^, equal to zero and resolving the problem amounts to deleting the 
k-th column from P and solving for p* in (3.1), i.e. P in. (3.1) is 
replaced by P with the k-th coltimn removed. Thus an optimal solution 
of the form (3.1) always exists where it is understood - that P represents 

/V 

the original P matrix with a certain number of columns, say r, re- 
moved where r,= the number of components in p* set equal to zero. Also, 
m is replaced by m-r. For the sake of discussion, in deriving the MSE 
for p*, it will be assumed that r = 0. 

Since P is a discrete random variable with z possible values, 
then the expected value of any function of P is easily derived, e.g. 
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E(P) = S q. P. 

. , 1 1 
1=1 


E(p-'p) = E q. p/p. 

i=l ^ 1 3. 


E(C'^C) - Z q, ct C, 
1=1 1 1 1 


where 


c. = (I- ^ j/) p.^ p. a~- j/) 

1 m 1 1 m 


We will assume here that P and e are obtained from independent 

y*V 

samples. Furthermore, e is obtained from a sample' of size n, where 
ne has a multinomial distribution, hence it follows that 


Cov (e., e.) = 
1 J 


e.(l-e.) 
X x' 


n 

-e.e, 



n 




1=3 


±^3 


Consequently, as with P, e has s =\ n / poss.ible 'values 
(say e(l), e(2) e(s)) with corresponding probabilities 


( n+m-l\ 


q^ (i 1, • • • ) s) 


where 


q,- = 


ni 


ti kj. 


i k *k^I ... k I *"1 


. e 


m 


and e(i) = 




'k^/n 


k /n 
m 


m 


m 
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Thus it is possible to determine the exoeoted vaIup nf at 
of P and e, e.g. 


^ /N 


s z 


/\ J\ /\ 


E(P e e P ) = 2 2 q' q. p, e(i) e(i)^ P.'^ 

i=l j=l ^ d d J 


Consequently, 


Also, 


*- 1 

y' . .m 


= E(p;> = ^ J + (I _ i jjT) y + + E3 


where 


I /V™ 

E^ = E(C P^ e) 


= E(C P ) E(e) 


= [S q. C-Tp 7] e 
i=,l 1 1 1 


1 :t 

ra 


-^rr. 


E 2 =■- - E(C P" P) J 


/N /N 


= - - [ S q. ct P.'^ P.] 

m i^l 1 r X 


E 


3 = - E(C'^C) y 


+ 

- [ 2 q. C. C,] y 


i=l 


XXI 


MSE (p*) 

y 


E (p* - p)(P*-p)‘ 


E(p* p*^ - p pj”^ - p* p"^ + pp'^) 


function 



62 


Now, E(p* p* ) 


. {[jJ + C P (a-ipj) 


+ (I - i Jj’^- c'^Oyl X [i j’^ + (? - 1 / ?)PC'^ + y’^a - i Jj’'-CC'^) 1 } 

ui m . fli m 


P* can take on z*s possible values considering all possible 

valued for P and e. 

Thus, 


E(p* Pf ) - S 2 qr q, f.. - w 

^ ^ i=i 3-1 J y 


where 


X ^ + (e(i) - i J^p J) P ct + (I - - - C.C."’’)] 

'w. JJ 


Thus MSE (p*) = W - p Up'^ - yp .p^ + pp^ 


and MSE (p* .)=w..+p.-2u.p. 

yx XX X pi ^x 


where (i) p*. = i-th component of p* 


•yx 


(ii) p^ = i-th component of P 
(iii) Pp^ = i-th component of p 


To illustrate the derivation of MSE (p*) consider the case for 
m=2, n^==2, n2=2, n=3, i.e. 


P = 


e = 



X, y = 0, 1, 2 


k = 0, 1, 2, 3. 



Referring to the previous discussion, the number of possible values of 
^.33 

R is z = ( 2 ) ( 2 )" the number of possible values of e is 

4 

s = Assuming that an optimal solution is obtained using the 

/s 

entire P matrix (for same vector y) , then 


U = E (p*) = i J + (I - i JJ^) Y + E E. 
^ ^ i=l ^ 


where, for example. 


and (q q), ?2 (q - (° °) 

H " P^Cl|l) [1 - pdlDl^-Cj) P°(2|2)[l-p(2|2)]^ 

% 

% = P°(l|l)tl - p(l|l)]^ • (o)p^(2|2)]°. 


Thus, for example, cj = [(I- | j/) (J J) (I - | 


T 

So, p* p* can assume z*s. = 36 possible values,, and the MSE (p*) 
given by 


MSE (p*) =: 

y 


4 9 

Z Z 
i=l j=l 


^i 




'F.. - 

13 


T 

py - 

p 




+ PP 


where, for example. 


(q) (1-e^) 


3 


92 = ( 2 ) P^(l|l)[l-P(l|l)]°. (J) p(2|2)[l-p(2|2)] 
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^2 = {-2 J + <=2^ ('l ?) [(?) - i (I i) J] + [I - i jj" - C^^CJy} 


rl ,T 


xtf + [(0 3) - i jT( 2 0„ (2 1) ^ - 1 jjT . c c +]} 


VII. CONCLUSION 

It has been shown that a vector p=p* exists which minimizes 
I 1 ^p~ ® 1 I subject to J p=l and p^O. Using the Iterative Search 
Technique (Nelson's procedure), the form of the solution was derived 
and was given by (3.1). Two methods of determining the solution' vector 
were derived and illustrated by several examples. The modified Simplex 
Method appears to provide the most "efficient" solution to the problem 
and can be easily coded for a digital computer. The two methods provide 
identical (and -unique) results when P is of full rank. When P is 
less than full rank, there will exist no unique solutionj however, for 
some vector y, the Simplex result-is- identical to the result using 
Nelson's procedure. The MSE for p* was derived in Section VI. Note 
that the expression for MSE (p*) also provides the MSE for the solu- 
tion vector provided by the Simplex Method since, for some vector y, 
the two methods produce identical results. 
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0. ABSTRACT 

The sensitivity of several proposed estimators of the 

mixture proportions a = defining the normal" mix - 

m 

ture density p(x;a) = E’ ct^p, (x) are investigated when the 

k=l ^ - 

component densities Pj^ are subjected to changes in location. 
The particular deviations studied are motivated by an applica- 
tion of this model to crop acreage' assessment using satellite 
multispectral sensor data. 


1. INTRODUCTION 

A current' problem in the NASA Earth Resources Program is 
the' application of mixture proportion estimators to crop acre- 
age assessment using multispectral sensor data from satellite 
platforms. The Earth Resources Technology Satellite (ERTS) 
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Department of Mathematical Sciences, University of Tulsa, 
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records reflected (or radiated) energy in four spectral wave 
bands corresponding to square 80 meter plots on the ground from 
an altitude of approximately 500 nautical miles. In an agri- 
cultural region training areas (labeled data) , consisting of 
known crops, are used to model or estimate the probability 
density functions of the existing crops. Then these density 
functions are used to construct a mixture model of a nearby 
recognition area in which no labeled data exists. That is, 
the mixture density of the recognition is given by 

m 

p(x;a) = E a.p, (x) (1) 

k=l ^ ^ 

where Pj^ is the component density and the proportion of the 
til 

k crop in the recognition area. The proportion vector 

T ■ ^ 

d = (otT/...,a^) must satisfy the constraints Z a, = 1 and 
1 m , T k 

k=l 

for k=l,...,m where m is the number of crops. In this 
application is interpreted as the acreage proportion of 
crop k in the recognition area. It is assumed that the compo- 
nent densities p^^ and the number of components m of the mixture 
are completely specified and a is the only unknown parameter. 

Several estimators of a have been proposed. [2, 4, 5, 6] for 
this model. (See [1,3,7] for a discussion of the mixture esti- 
mation problem in a more general setting, i.e. when the densi- 
ties Pj^ are not completely specified.) The following four 
estimators were chosen for this study. 

i) Classification (CLASS) - is the proportion of 
points in the recognition area which are classified into the 
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k class -by a .maximum likelihood classifier defined by the 
training densities pj^. 

li) Maximum Ii ik e 1 x hood (MbE) ~ a is the vector which 
maximizes the likelihood function defined in (1) . 

iii) Moment (MOM) - a is the vector which fits the mix- 
ture moments with the empirical moments of the unlabeled data 
in the least squares sense* 

Minimum Chi-square (MIX) — a is the vector which 
fits the percentiles of the marginal mixture distribution 
functions with the corresponding empirical percentiles of the 
uniabeled data set in the least squares sense. 

A detailed description of each estimator is given in the 
appendix. The estimator proposed in [5], which modifies the 
classification estimate by using the knowledge of the confusion 
matrix associated with the -classification rule, was not, 
included in this study since its behavior would exactly paral- 
lel that of the classification estimator if the confusion 
matrix were based only on the training area data. If addi- 
tional information was known in the recognition area, then it 
is' felt that this method would be worth considering. 

In this_ application it is common for the components den- 
sities of the crops in the recognition area to deviate from 
the model which was based on the training area, even though 
the two areas are close geographically. Por example, if the 
planting times of one crop, say wheat, were different in the 
two areas, other crops remaining fixed, then the difference in 



70 


maturity would cause a shift in the wheat distribution and the 
model would be inaccurate. If the data acquisition times were 
different, say by a day, then different sun angle, and atmos- 
pheric conditions would cause a shift in ail crop distributions. 
This study was undertaken in order to determine how each- of 
the above estimators behave under such deviations of the model. 
It is assumed that no labeled data exists in the recognition 
area, otherwise the crop distributions would be adjusted to 
reflect any errors in the model. 


2. SIMULATION STODY 
2.1 Description of the Simulation 

The aim of this simulation study was to evaluate the 
effectiveness of the proportion estimators when the data in 
the recognition area has been "shifted" from the training 
area. The simulation was made as simple as possible in order 
to remove as many extraneous factors (sampling error, corre- 
lated variates, etc.) as possible. The study may be outlined 
as follows; 

(a) Generate a random sample from pop- 
ulation IT., where ~ M\m(y.,cfl) for i=l,2,3 

X X 

when n^, and a are known fixed parameters. 

(b) Train the proportion estimators using the train- 
ing segment. 

(c) Create a new data set (recognition segment) 



Y . ..Y where Y - x' ^ + dv. when N = Z n. , d 
is a known scalar and is a fixed direction 
/ector for population ir^. 

(d) Determine the proportion estimates in the new 
recognition segment and evaluate the estimators 
using the sum of squared error deviation from 


the ground truth. 

In this study = (30/30)^, = (40,40)'^, = (40,20)^ 

and a - 7. Using these parameters we defined two' experiments, 

(1) n^ = 180, n 2 = 75, n 3 = 45, (2) = -^2 " ^3 " 

both experiments we defined the following three sets of direc~ 
tion vectors : 

( 1 ) v^^ = ( 1 , 0 )'^, V 2 = ( 0 , 0 )^, V 3 = ( 0 , 0 )'^ and 
d = 3,6,9. 

( 2 ) . v^ = ( 1 , 1 )'^, v^ =_( 0 , 0 )'^, V 3 = ( 0 , 0 )"^ and 

d = 3,6,9. 

(3) Vj^ = V 2 = V 3 = (1,0)'^ and d = 3,6,9. 

Each case .was repeated three times for each experiment using 
an independent data set. c. a. 7 


results of the simulation have been summarized in 
Figures 1-6 and Tables 1-2. The values found in both the 
figures and tables represent the average mean-squared error 
and crop proportions , taken over the three data sets . In 
Figures 1-6 the mean-squared error is plotted against the 
distance the recognition area was shifted. Tables 1-2 give 
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TABLE I 


CASE 

DIST. 

POP. 

CLASS 

MLE 

MOM 

MIX 

ANS. 

1.1 ■ 

0. 

I 

.5956 

.6036 

.6027 

.6000 

.6000 



II 

.2533 

.2491 

.2431 

.2500 

.2500 



III 

.1511 

.1472 

.1494 

.1500 

.1500 


3. 

I 

.5600 

.5773 

.4368 

.4640 




II 

■ .2711 

.2630 

.3216 

.3155 




III 

.1689 

.1598 

.2416 

.2205 



. 6. 

I 

.,4122 

.4560 

,2565 

.3536 




II 

.3422 

.3267 

.4035 

.3663 




III 

.2456 

.2173 

.3400 

.2801 



9. 

I 

.1811 

.1954 

.0677 

.2052 




II 

.4300 

.4493’ 

.4886 

.4305 




III . 

.3889 

.3553 

.4437 

.3643 


1.2 

3. 

I 

. .4878 

.5188 

.4412 

.4525 




II 

.3622 

.3342 

.4094 

.4474 




III 

.1500 

.1471 

.1494 

.1002 



6. 

I 

,1644 

.1364 

.2626 

.3034 




II 

.6878 

.7153 

.5877 

.5676 




III 

.1478 

.1483 ” 

.1497 

.1291 



9. 

I 

.0122 

.0005 

.0727 

.0661 




II 

.8411 

.8500 

.7723 

.7639 




III 

.1467 

.1494 

.1500 

.1700 


1.3 

3. 

I 

.5567 

.5745 

.3023 

.4140 




II 

.2711 

.2626 

.3825 

.3370 




III 

.1722 

.1628 

,3153 

.2488 



6. 

I 

.4089 

.4651 

.0072 

.,2707 




II 

.3422 

.3125 

.5171 

.4039 




III 

.2489 

.2224 

.4757 

.3254 



9. 

I 

.1778 

.2635 

.0000 

.0953 




II 

.4300 

.3939 

.5350 

.4815 




III 

.3900 

.3426 

.4650 

. .4232 
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TABLE II 


CASE 

DIST. 

POP. 


iviLE 

MOM' 

MIX 

2.1 

0. 

I 

.3322 

.3357 

.3260 

.3333 



II 

.3333 

.3324 

.3390 

.3333 



III 

.3344 

.3320 

. .3350 

.3333 


3. 

I 

.3133 

.3148 

.2307 

.2824 



II 

.3433 

.3-458 

.3827 

.3588 



III 

.3433 

.3394 

.3866 

.3588 


6. 

I 

-.2300 

.2272 

.1304 

.2223 



II 

.3900 

-.3945 

.4283 

.3898 



III 

.3800 

.3782 

.4413 

.3879 


9. 

I 

.1017 

.0867- 

.0070 

.1407 



II 

.4417 

.4625 

.4797 

.4342 



III 

.4567 

.4510 

.5133 

.4251 

2.2 

3. 

I 

■ .2656 

.2684 

.2335 

.2731 



II 

.4000 

.39-96 

.4317 

.4144 



III 

.3344 

.3320 

.3349 

.3126 


6. 

I 

.0756 

.0532 

.1342 

.1955 



II 

.5900 

.6137 

.5310 

.4819 



III 

.3344 

.3331 

.3348 

.3226 


9. 

I 

.0022 

.0008 

.0296 

.0628 



II 

.6644 

.6674 

.6326 

.6074 



III 

.3333 

.3319 

.3341 

.3297 

2.3 

3. 

I 

.3122 

.3133 

.0173 

» 

.1769 



II 

.3444 

.3485 

.4787 

.4168 



III 

.3433 

.3381 

.5039 

.4063 


6. 

I 

• .2289 

.2549 

.0000 

.0913 



II 

.3911 

.3790 

.5009 

.4610 



III 

.3800 

.3661 

.4991 

.4476 


9. 

I 

.1056 

.2124 

.0000 

.0000 



II 

.4400 

.3886 

.5079 

• .5081 



III 

.4544 

.3991 

.4872 

.4870 


ANS. 


.3333 

.3333 

.3333 





the crop proportions found using the proportion estimators at 
the various distances. 


3 . .CONCLUSIONS 

Since the training model consists of component classes 
in which the interclass confusion is small, all estimators do 
well when the model is exact (d=0) . Further- the behavior of 
the estimators is similar under both experiments. That is, 
the true proportion vector does not seem 'to be a definite 
factor in the sensitivity analysis. Now' consider each case 
separately. In case 1 the CLASS, MLE, and MIX estimators 
perform similarly with the first two better for d = 3,6. The 
MOM estimator is uniformly the most sensitive to the model 
deviations. In case 2 when the first class is shifted toward 
the second the likelihood based estimators (CLASS and MLE) are 
worse as the 1st class crosses the classification boundary, ' 
and the MOM and MIX perform similarly. In case 3 the MOM and 
MIX estimators are more sensitive than the CLASS and MLE. 

In conclusion, based on the types of deviations considered 
here, it appears that the ordering of the four estimators, 
according to the degree of sensitivity, which is suggested by 
this experiment would be (CLASS, MLE) ^ MIX ^ MOM. It is also 
apparent however that the particular type of shift deviation 
from the model may give a different ordering. Therefore, if 
the suspected deviation is known to be of one particular type 
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run 

or direction, then a spe-oiaAafaed experiment should be'^^aiy^to 
investigate the sensitivity under this alternative. 


4. APPENDIX 


4.1 Classification (CLASS) 
Define 


i£ p3^(X) > Pj(X) 

I 0 otherwise 


for k =* 1, . . . ,m. Now the CLASS estimate is given by 


1 , 

■ “k “ N ^k ^ “ l/...,m, 

where X^,...,Xj^ is the sample of unlabeled data, ccj^ is simply 
the proportion of the sample which is classified into the kth 
class by the maximum-likelihood classifier. 

4.2 Maximum-Likelihood (MLE) 

A necessary condition for a to be a maximum-likelihood 
estimator of a is that 


“k = I ij,P^.(Xj)/p(X.;a) 

3 t 

tot' k = . ,m. See [6] for. a more detailed discussion of 

this estimator. 

4 . 3 Moment (MOM) ' 

For each class k let p^, j = 1 , . . . , denote the 
first and second order noncentral moments. (Here n is the 
dimension of the multivariate observations and p^ is the mean 
of the jth variable for j = l,...,n and one of the distinct 
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elements of the noncentral second moment matrix for j = n+1, 
^ , n (n+1) X , . , . . 

2 — r* Xj he the correspondxng sample moment 
from the unlabeled data for j = 1,..., n+— ^ , Now con- 
struct the system of, equations 


ot, y . 
^3 


, n (n+1) 


where the weights w^ are proportional to the variance of the 
sample moment . Now the MOM estimator is defined to be the 
constrained least squares solution of this linear system. 


4.4 Minimum Chi-square (MIX) 

For each class k let be the marginal distribution 
function and let F^ be the j^h marginal empirical distribution 


function of the unlabeled data for j = 
the 100^^ percentile of F^ for i = 1. 


1, . . . ,n. Let Y . . be 

-XJ 

-.,s. Now construct 


m -ck,-- . 

J k-1 

for i = l,...,s and j = l,..,,n. (In this experiment s = 9 
and n = 4) . The MIX estimator is defined to be the constrained 
least squares solution to this linear system. 

4.5 Odell-Chhikara (0-C) 

Let P = t>e the confusion matrix defined by the max- 

imum likelihood classification procedure where 

p£j = Pr‘[Xi(x) = ijxeTTj]. 

Since E(a) = Pa, where a is the estimator defined by CLASS, the 
0-C estimator is given by 
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a 


when P is an- estimate of. the confusion matrix. 
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ABSTRACT 


Lefp,/ . . . . p be multivariate density functions and let a = (a,, . . . ,a ) 
be a proportion vector defining the mixture density p(x,ix) = °'-.p.(x) , where 

2^^ a. = 1 and ct. > 0 for i = 1/ . . . ^ m. This paper investigates the problem of evalu- 
'ation of the maximum-likelihood estimate force. An acreage estimation application is 
presented using remotely sensed data. 
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1 . IntroducMon 

Let « « • / ir^bem pattern classes having distinct multivariate probability 
density functions p^, . . . , p^ respectively. Leta= . . . , be the proportion 

vector defining the mixture density 

m 

(1) p(x;a) = Z ^aj^Pj^(x) 

where *^ 1 ^ = 1 and > 0 k = 1 , . . , , m. Let 56 = be a random 

sample drawn from the mixture density p. A necessary condition for a vector 

peS=fYeR"’ : Sy. = 1 ; Y. > 0, I = 1 , . . . ,m} 
to be the maximum-likelihood estimator (MLE) of a in (1) is well known (see [2,p.192]) . 
However, the problem Is usually considered In the context of the general mixture problem, 
where the component density functions and the proportion vector are simultaneously estimated 
from the mixture sample X, In this paper the component densities are assumed to be completely 
specified. This is an appropriate mathematical model even when labeled data, independent 
of3o, is used to estimate the component densities prior to drawing the mixture of unlabeled 
sample. The following is included for completeness. 

Theorem 1 A necessary condition for P e S to be a MLE of a in (1) is 

( 2 ) « 

for k— 1, , , ,,m. That is, the MLE must be a solution of the fixed point equation 

(3) • p = G(p), 

where G is a vector valued function defined component-wise by the RHS of (2) . 

Proof; The log-likelihood function 

N m 

I 0) = S log S p.p.(X.) 
j=l i=l ' ' ^ 

IS concave and the constraint set S is compact and convex. Hence, P s S maximizes 
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I on S jf and only if 7} ( 3) (y - P) ^ 0 for all Y S . That is, if and only if 


ai 

9P, 


(P) < ^1 (P) P 


• • • ,m* 


It is easily verified that 7|(P) P = N and thus P maximizes I on S if and only if 

M) _L IL (B1 = _L s . , 

N 3 ^ N j=1 p(Xf 3) i ’ 

k= 1, , , , , m, with equality whenever Pj^> 0. Multiplying both sides of these inequalities 

I 

by the corresponding Pj^ yields the required necessary condition. 

2, An Iterative Method for Obtaining the MLE . 

The fixed point equation (3) suggests that a possible method of obtaining the MLE, 
say a*, is to iterate G and form the sequence of successive approximations P*^ = G"^(P) , 
where Pis the initial guess of a^*’. The purpose of this section is to provide partial theoretical 
justification for its use. 

Let denote the relative interior of the constraint set S . We will assume that 
the Nxm matrix P = (p. (X.)) has rank m so that the Hessian of the log-likelihood 


fun ction 


/ p^(X.)^ P,(X.)P^(X..) 


(5] 


N 

H(P) = - 


\pJX,)p,(X.)...pjX.) j 


p(X. , P)' 


is negative definite for all P e S and consequently the MLE CC* is unique. Moreover, 
with this assumption, both S and are invariant under G , In practice, N is very much 
larger than m, so that the assumption that P has rank m is almost always satisfied. 

Note that G has fixed points other than cc* . |n particular, each vertex of S is 
fixed by G . The next theorem shows that G is unstable near these extraneous solutions 
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of the likelihood equations. 
Theorem 2 


Proof: 


lf8eS®and^= lim G"(P),then 
n ** £» 

If P is not a MLE, then since P= G( P) it follows from (4) 


il 9 1 ■' 

"N 3^ ( P) ^ 1 for some k such that Pj^ = 0. Since P= lim P , for large n we 


have 


1 91 


N 9p, 


{^) >1 


and pj^ > 0, since Pe S , Therefore, 

0^"^^ — eP ^ ^ I /R*^\ V. a*^ 

^ • TT TO 

k 

for sufficiently large n. It follows that P^^ cannot converge to 0 = Pj^ , a contradiction. 

The next theorem shows that under certain restrictions, G is a local contraction at 
the MLE a*. That is, for some norm on there exists ^ > Q and k, 0 < k < 1 . Such that 


|ja* - G(P) Ikk II -^P II 

whenever llct* - p j) < e , Thus if P is sufficiently near a*, the sequence P*^ = G^(P) 
converges geometrically to a'**'. The proofs of Theorems 3 and 4 are given in the Appendix, 
Theorem 3 If the rank of P is m and the MLE a^^is in S^, then G is a local 

contraction at a*. 

For the two class case, we have the following somewhat stronger result. 

Theorem 4 If m = 2 and P e S^ then G^(P) converges to a*. 


3, Choosing a Starting Vector: 

It is clear that any starting vector P should be in S^. If no other information is 
available, then a naive starting vector would be P = — ( 1, . , . , 1)^. If the com- 
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component densities are widely separated, then the probability of error P of the 
maximum-likelihood classifier is small and the proportion of observations classified into 
each component class would be approximately the MLE. That is, let 


and 




T 

0 


if Pj^(x) > pj(x) for j + k 
otherwise 




1 


N 


^::,)Ck(xp 


k = i. 


m. 


Even if IS not negligable, P should serve as a very good starting vector pro 
viding 3 e S® , 


4. An Example ; 

In this section, the previous results are opplied to the problem of estimating crop 
acreage from satellite data. The data used was taken by the multispectral scanner aboard 
the NASA launched. Earth Resources Technology Satellite (ERTS) , This sensor records 
reflected (or radiated) energy in four spectral wave bonds corresponding to square 80 meter 
plots on the ground from an altitude of approximately 500 nautical- miles. Five crops w.ere 
identified and their density functions were estimated from labeled data using a multivariate 
normal model. In Case I , all five crops were represented by at least 10^ of the total 
mixture (unlabeled) sample of 1000 points. In Case II, the fourth crop was retained in- 
the model, but deleted from the mixture sample. 

The seven starting vectors which were tried in each case are listed in Table I, along 
with the MLE in each case. All starting points converged to the MLE in both cases under 
the iterative procedure. The first two starting vectors are those proposed in section III . 
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The last five were chosen near vertices of S to illustrate the convergence rate when a 
"poor" choice of starting point is used. 



MLE 

1 

CROP 

2 

3 

4 

5 


I 

.2937060 

.3616114 

.1581606 

.1000086 

.0865135 


II 

.3224670 

.4115446 

.1745424 

.0032386 

.0882074 


P 

1 

2 

3 

4 

5 

1 

1 

.278000 

.278000 

.170000 

.106000 

.168000 


II 

.304444 

.306667 

.186667 

.023333 

.178889 

2 


.20 

.20 

.20 

.20 

.20 

3 


.01 

.01 

.01 

.01 

.96 

4 


.01 

.01 

.01 

.96 

.01 

5 


.01 

.01 

.96 

.01 

.01 

6 


.01 

.96 

.01 

.01 

.01 

7 


,96 

.01 

.01 

.01 

.01 


TABLE 1 Starting Vectors 


Iri Table 2 the number of iterations required for 2,3, and 4 place accuracy (using 
the sup-norm) are noted for each starting vector. That is, the table entry is the number 
of iterations n for which 

max I a!" - G? (B) I < .5x for k = 2,3,4 . 
i=:l 5 ^ ■* 
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TABLE 2, Iterations needed for k place accuracy 
using starting vector P 


For these two examples the iteration procedure appears to be very stable with only starting 
vectors 3 and 6 requiring more than 3 iterations more than the best starting vector. 
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APPENDIX 


Proof of Theorem 3: 

0Gj^ 

‘the mxm matrix ( va --(P) ) • 
0 Pj 


The Frechet derivative G'(P) of G is represented by 
From (2) it follows after a brief calculation that 


( 6 ) 


G' (3) = diag( 


1 

N 




diagCe,^) H(W 


'where H(3) is the Hessian of I given in (5) . Since G'(3) is a continuous function 
of 3, it follows from the mean value theorem that G is a local controction at of , 
if, with respect to some norm on r"' . 

l!G'(a*) 11< 1 

where llG‘(a*) lt= sup llG'(a*)3il. By [2, p, 46] this is true if and only If the spectral 

IIPIM . 

radius p(G'((X^) is less than 1, Since Ct* is the MLE and a*sS , it follows from (4) and (6) 
that 

G-(of) = 1+-^ diag(a*) H(a*). 

Since the eigenvalues of diag (a* ) H(a*) are negative, p(G' (a*) ) < 1 if and only if 
p(diag(aj|^)H(a^ ) <2N, The entries in the matrix diag (a*) H (a*) are all < 0 and it is 
-easily verified that 

diag(o^)H(a*) a* = -Na*. 

It follows by Frobenius' Theorem, [5,p.49], that the spectral radius of diag (a* ) H (a*) is N » 
Hence- p(G' (cc^ ) < 1 and the proof is complete. 

> Proof of Theorem 4 ; We will show that whenever 3 ’=• S^ and 34= cc*, then 

(7) IG ^(3) -d*K| 3^ -a* I 

from which the theorem follows. From the concavity of I we have 
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0 <v| (g)(a«-P) (p) - |i (P)). 

Hence, P^<o.* If and only if (P) >-~ (P) . Since P^ —— + P 2 ^ = N, 

P^ <a* if and only if (P) > N . Similarly P^ > ci* if and only if (P) < N . 
Assuming that / 

and (7)' follows if (P) < a* ♦ i.e., if ^^(G{P))>N. 

But, 

3 1 Pi 

-^(G(P))- ^2 ^ G^(P)p^(X.)+G2(P)p2(X.) 

N P/X) 

_ 2 * J 

j=1 P,P,(Xj)-Frlf;P2P2(><i>-K8^ 

N p (X ) 

y 5;; » J 

i=, (b^p,(X.)-+-S2P2(X.))-L^ 

= N . 

Therefore, G^{p) converges to a* , 
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Some Results on Randomly- 
Missing Data in Discriminant Analysis 

by T.L. Boullion 


Several alternatives are available for handling randomly 
missing data from multispectral scanner measurement acquired over 
an agricultural area. One such procedure is described in [1] . 

In this report/ other methods are investigated and compared using 
the asymtotic unconditional probability of correct classification. 

The five methods considered are (1) Use only the complete 
observation vectors. All vectors with any missing components 
are discarded. The linear discriminant function is calculated 
in -the usual manner with only the-complete observation vectors. 

(2) Use all sample values in computing means and variances for 
each variable, but only complete pairs to compute covariances. 

Then the linear discriminant function is calculated with the 
sample covariance matrix formed from these statistics. (3) Compute 
means based on all available values and substitute these mean 
values for the missing values. Calculate the discriminant 
function from this completed set of vectors. (4) Use the 
estimate from the regression equations, regressing the missing 
values on those which are not missing, to complete the- vectors 
with missing components. The linear discriminant function is 
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calcuiatea trom tnis set of completed vectors. (5) Use the first 

principal component from each of the two sample matrices to 

estimate the missing values as follows-: Let X, and be the 

)xr 


pxni. 


pxm. 


data matrices whose columns are the observation vectors on two 

crops of interest. Replace i = 1,2 where 

y^v “ “ X . ) //s . . for known values and y =0 for missing 

3 ^ 3 ^ 3 33 

* th 

values. X- xs the sample mean of the j variate and s. . is the 

til 

pooled variance of the j variate. The coefficients of the 
first principal component of are then .obtained, .say where 

/ . • . ^ is the eigenvector of unit length associated 

X p 

T 

with the largest eigenvalue of Missing values in are 

replaced by the coordinate of the nearest point on the first 

principal component. That is, for each data matrix, Y., is re- 

P . ^ 

placed by a„q . , where = 

X J X 


Yjj^qj. After all missing values 


j=l 

are estimated, Y^ and Y 2 are transformed back to their original 
units X^ and X 2 / and the usual discriminant function calculated. 

To compare the methods, -the actual probability of correct 
classification using the usual discriminant function based on a 
single pair of samples is used. The expression is 


P = i 0 ( 






/lC^"^-X^2)jT3-l^g-l(-(l)_-(2)j 


( 1 ) 



97 


In the above y ^ , i = 1-2 are the population means, 2 is the 
population covariance matrix, is the vector estimate of y 

from the particular missing value method used, and S is the 
pooled estimate of 2 from the particular missing value method. 

The asymptotic limits of X ^ X^^^ and S for each missing 

value method were obtained and substituted into (1) to obtain 
numerical values for the asymptotic probability of correct 
classification which equals E^. The variances were taken equal to 
one without any loss of generality and all correlation coefficients 
were taken equal; hence 2 = R = (p) . 

Methods (!) and (2) attain the maximum probability of correct 
classification, hence only methods (3) , (4) and (5) were compared. 
The Mahalanobis distance A = y^R between the two populations 
was taken to be 4, and the proportion of missing values was taken 
to be m = .2. The correlation p was restricted to being greater 
than -l/(p-l) so that the equi correlation matrix would be positive 
definite. 

Partial results are given in table I. 


TABLE I 


Asymptotic Probability of Correct Classification, 


E 

P 


p method 

.8 


P 

^ ^ 

(a) y = (d,0,. . . ,0) ^ 


2 


(3) .8330 


. 8397 


. 8411 


.8413 




(4) 

.8399 

.8400 

.8411 

.8413 

— 

— 

— 


(5) 

.8413 

.8405 

.8388 

.8370 

— 

■ — , — 

— 

3 

(3) 

.8370 

.8410 

..8411 

.8413 

; 

.8406 

.8299 


(4) 

.8410 

.8409 

.8411 

.8413 

— 

.8407 

.8388 


(5) 

.8413 

.8410 

.8400 

.8381 



. 8325 

. 8022 

4 

(3) 

.8387 

.8404 

.8411 

.8413 

— 





(4) 

.8412 

.8411 

.8412 

.8413 

— 

— 




(5) 

.8413 

.8412 

.8405 

.8388 

— 


— 

8 

(3) 

, 8406 

.8410 

.-8412 

.‘8413 

.8376 





(4) 

.8413 

.8413 

.8412 

.8413 

. 8374 




(5) 

.8413 

.8413- 

.8411 

.8400 

. 8292 

— 





(b) 

y = (d,d 

,0)^. (d,d 

o 

o 

(d,d,d 

/ d , 0 , 0 , 

0,'0)^ 



for p =f 3 

,4 and 8, 

respectively. 



3 

(3) 

.8300 

.8390 

.8410 

.8413 

— 

. 8408 

. 8359 


(4) 

.8404 

' .8402 

.8410 

. 8413 


.8410‘ 

.8405 


(5) 

.8413 

.-8406 

.8390 

.8372 

— 

.8332 

.8158 

4 

(3) 

.8348 

. 8394 

.8409 

.8413 

— 

.8391 



(4) 

,8410 

.8407 

.8410 

.8413 

— 

-.8400 

— 


(5) 

.8413 

. 8409 

.8397 

. 8373 



. 8265 


8 

(3) 

. 8370 

.8396 

.8406 

.8413 

,8374 



— ^ — 


(4) 

.8412 

.8409 

.8408 

.8413 

.8392 



— 


(5) 

.8413 

.8411 

.8404 

.8374 

, 8233 


rn. 


A 


2 , 


Pi 


.2, R 


/I P. 


8413, m 


p • ■ « p X 
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= .8413 is the highest attainable probability of correct 

T 

classification. For t = (d,...rd) , Ep = p^ for all entries. 

Though the asymtotic performance among the missing value 
methods have slight variations, the differences' of Ep are not 
svibstantial. Thus,, it is concluded that the treatment of missing 
values in discriminant analysis must be studied for small and 
medium size samples. 

With this in mind, a pilot study was conducted, comparing 
methods (1) and (4) in the two population discrimination problem. 

This missing data problem is such that there are numerous 
variables which affect the outcome. This study focuses on two 
of these; 1) The percent of complete versus incomplete vectors 
and 2 ) The discriminatory power in the mean vector relative to 
the particular component or components which are missing. To 
gain insight into the effect of these factors several simplifying 
conventions were adopted. First of all, only two class patterns 
were specified at any time. The first class of vectors consisted 
of those which were complete and the second class consisted of 
vectors with one particular component missing.- The second con- 
vention was that two simple but strong covariance structures were 
used throughout in order that the strongest statement with respect 
to the use of method (1) and (4) could be obtained. The first 
covariance structure exhibited near perfect correlation between 
the component which was to be missing in the second class pattern 
and one of the other components; all other covariances were zero. 
The first covariance structure for p = 4 might be represented as 
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/ 1 

? ~ 0 0 1 
\ 0 0 0 1 

The second covariance structure had the correlation between 
the component to be missing in the second class pattern and all 
other components to be 1/^' (i.e. the multiple correlation 
coefficient is arbitrarily close to one) ; all other covariances 
are zero. This structure might be represented by 

/' . 

1//J 1 

^ "" 1//5 “ 0 1 

\ 1//J ' 0 0 1 

Thus, viewing the missing dat^problem in terms of filling 
in the missing components with their regression estimates, given 
the components of the vector which are not missing, the above 
convention adopted in this study will provide for the maximum 
increase in information brought about by the inclusion of the 
partial data vectors . The mean vector for group 1 either had all 
components equal and nonzero or otherwise the mean associated with 
the particular component in the second class pattern which was to 
be missing was nonzero, and all other 'means were zero. The mean 
vector associated with group 2 was always the zero vector. 

The actual simulation methodology will now be described. It 
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was arbitrarily determinecj that p = 4. Mean vectors and. a co- 
variance matrix were 'specified and training samples of sizes 

= 100 were generated from each of two groups such that a 
specified percentage of the vectors were complete and the rest 
had a particular conponent missing. The well known Bayes linear 
classification rule was determined using only the complete vectors 
in the training samples and the method (4) was employed using all 
■tdie data. Next, samples of size N^ = N 2 = 100 were generated 
from the two groups and the vectors classified by both rules 
into one of the two groups and the n\imber of misclassifications 
were counted. The entire procedure was then repeated until 25 
simulations had been performed. It was determined that 25 
simulations per particular set of mean vectors, common covariance 
matrix, and training sample design was sufficient to ascertain 
the relative performances of the two rules. The actual simulation 
results are summarized below. 
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Summary, 

xr seems logical that the inclusion of additional information 
into a classification rule might reduce the probability of 
misclassification; however^ it must be remembered that given a 
set of parameters (i.e. mean vectors and covariance matrices) and 
the Bayes linear discriminant rule, the probability of misclass- 
ification is fixed and that the classical Bayes rule , regardless 
of sample size, can be used to provide unbiased estimates of such. 
Hence, one should expect the increase in performance resulting 
from the imput of additional information (i.e. data, whether it 
be complete or incomplete) to manifest itself in the variances 
of the associated estimates of the probability .of misclassification. 
However , the estimated variances of the estimates of the pro- 
babilities of misclassification as computed in this simulation 
study do not in general tend to support this claim with respect 
to the inclusion of incomplete data. Even with the exceptional 
covariance, structures specified in this Study, the only evidence, 
and it is extremely weak, found in support of smaller variances 
associated with method (4) is found when the percentage of incom- 
plete vectors is very high (i.e. 90%) and when the population 
mean vectors differ in every component. Further interpretation 
of the above results is left to the reader. It is felt that 
the common and known covariance matrix is at least partially 
responsible for the results obtained in this study since the 
estimation involved in the development of the two rules is only 
with respect to estimating the mean vectors associated with the 
two groups. 
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ESTIMATION AND CLASSIFICATION WITH INCOMPLETE DATA 
Thomas L. Bouillon, Benjamin S. Duran and Patrick L. Odell 

1. Introduction and Summary 

In this paper we consider the problem, of missing data from multispectral 
scanner measurements acquired over an agricultural area. Frequently, due 
to partial cloud cover and other factors, not all elements of the multi- 
variate observation vectors are meaningful on every occasion. Since these 
occur randomly, this information is taken into account to estimate para- 
meters using the incomplete as well as the complete data vectors. 

Assuming a multivariate normal for the distribution of the multi- 
spectral scanner measurements we develop expressions for the estimators 
of the mean vector and covariance matrix based on all the data. Also, 
expressions are given showing the gain in precision which may be obtained 
by using incomplete as well as the complete data. 

Since this phenomenon of missing dat^is present in the training 
samples as well as for vector observations to be classified later, we develop 
a maximum likelihood scheme for classifying an observation vector in one of 
two multivariate populations with unknoira means, but known covariance matrices. 

2. Description of the Model 

Assume a training sample of size N is taken from a p-variate normal 
distribution, but some of the p-vectors of observations have randomly oc- 
curring missing entries. In remote sensing applications, each observation 
vector X is of the form X = [X^, X^, X^, X^], where X^(4xl) represents a 
multispectral scanner measurement taken at time i. (For instance, each X^ 
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could be an observation on an agricultural unit at various growth stages 
for a crop of economic interest, such as wheat.) In this case, missing 
data occurs whenever a complete subvector is missing, or can be identi- 

fied to be cloud cover over the unit. Thus, there are 2^-1 (in general, 
there are 2^-1) possible sets of partial data vectors. Let be the set 
consisting of all complete data vectors. Let denote the i-th set of 
Partial data vectors arranged in an arbitrary' order starting with To 

illustrate, let R be the set of all vectors with X. missing, R be the 
set of all vectors with X^ and X^ missing, etc. 

X^^(a=l, . . . ,m^) denote the (p^><l) observation vectors corresponding 

_ 1 “i 

R- (i— 1,2, ... ,2^—1) , and X.= Z X. , ■vdiere X. does not exist if m.=0. 

X iQ • ^ io(. i 2 . 

X a=l 

This leads us to assume the following statistical model: 

^la distributed as a p-variate normal with means y and covariance Z; 
^ia distributed as a p. -variate normal with mean y, and co- 

X 1 

T 

variance Z , where y = D,y and Z. = D.ZD., where D. is a matrix of 

X XX XXX X 

ones and zeros indicating which observations are missing. For instance, 

if R^ is the set of all vectors with X^ missing, then ' 

10 0 0 
010 0 
0 0 10 

»^24 
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and I . 



11 

h2 

^13 

^14 

21 

^22 

^23 

^24 

31 

^32 

^33 

^34 

41 

^42 

^43 

^44 



( 


^11 

^12 

ha 

z 

£ 

z 

21 

22 

23 

Z , 

z 

z 

31 

32 

33 


3. The Likelihood Equations I-nien Is KnoTO 

Having the training sample properly categorized in the sets R^(i=l,2 , . . , jk) 
(note k = 2^-1 if none of the are empty), R^ == • • • >^ 3 ^^ ^ 

R 2 “ ^^21*^22’ *’* *^2m ®tc., the likelihood functions of the sample can 

be written as; 

L = L 2 ^,L 2 > . . . ,Lj^, where 

m. 

X 


L. 

X 


p.m. ta. 

X X X 

(2tt) ^ |SJ 2 


- 1 (X. -y .)^ (X. -y!) 

^2 T xa X X xot X 

e 0=1 


K 


The logarithm of the likelihood function is thus , log L = Z log L . , where 

i=l 

1 1 
log — p^m^ log(2ir) loglz^[ - ~ tr(Z^TI^) 

X T 

where the matrix M. is given by M. = Z (X, -yJCX. -y.) = m,(Z. + H,). The 

a=l 

^ • >* - ^ • ys ^ rp 

1 X i 

matrices Z. and H. are given by Z . = — Z (X. -y.)(X. -y.) and 
1 1 xm.^xax xo x 

X 0=1 

rp /V El • 

^ 1 ^ 

~ (y -“y^- ) (p^-~y • ) > where y. = — Z X. . Thus, log L. can be written as 

1 ■ 1 1 X X X m^ X X 

1 1 
log \ = - 2 Pi*"! 1 osC2tt) - 2 ~ logh'ii " -p tr.(Z“ + \D and 



109 


8 log 
Sp“ 

X 


--a- 




a log L T / ^ L \ 

Since ^ j , we have ^ = - E m^.D^E. ’^(D_. jj~y . ) 


a log L. k „ 

J-. T' 


i=l 


XXIX X 


Differentiating log a second time with respect to yields 


a log L 

3 yT ay, 

X i 


— = But, the negative expected value of this is the portion 


of the information matrix corresponding to y . , say W , which is equal to 

’■‘i 


m,E ♦ Similarly, ^ — = *- Z m.D.Z.T). and thus the total informa- 

sy’^sp 1-1 ^ ^ 


X X 


T — 1 

tion matrix for y is W = E D.2, D.. This enables us to write 

u i.i 1 1 1 


a los L ,. . . _T._ 

—3^ y + S D^H y^ , 

i=l i 


(1) 


which upon setting -equal to zero and solving for y yields the maximum lilceli- 
hood estimator y for y(for E known) as 


k _ /V 

y = W,, E d: W,, y. . 
^ i=l ^ ^i ^ 


,-l 


The matrix is of interest since it is the as 3 nnptotic covariance 

A 

matrix of y. Also, if k=2 the estimator y becomes 
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n. 


V^2/ "2 


(yii-v^j) 


^12 " N ^^11 ^21^ 


) 


where is the sample mean of the j—th component -of y computed from vectors 


in R . The large sample covariance matrix is given by W~^ where 

y 

T _i ■ 

“ ^ 1 . ^ easily verified that W ■*■ can be written in 

the form: 

W“^ = |l - (D W"^ + W"^) 

y L y^L 2 2 y^, 2 2] 


The second term represents the gain in precision which may be expected if 
both groups of data are used as opposed to only the complete observation 
vectors to estimate y. Investigating the variances of each component of y 
we obtain the following: 

var(y^) = if • the i-th component of the data vectors is not 

— 9 

missing in R„, and var(y ) = (1 - •— R.) g ../m- otherwise. R, 

JN X XX X X 

is the multiple correlation coefficient obtained when regressing the 

i— th variable on the variables corresponding to R^ . 


^ • The Likelihood Equations Tflien S is UnknoTO 


It is convenient to display the elements of as a column vector 


whose elements are ordered as the columns of I. To illustrate, let 


•^11 ^12 ^13 


E -I 


C 22 ^23 > ^ ^ 23 ^^ 33 ^- Thus, a is a 


33 



lU 


vector of length j p(p+l) -defined by a.= (a_; l£i_<j =l,2,...,p). The 
vector cr^ of length P^Cp^+1) will represent the corresponding column 
array of i=l,2,...,k. To relate to a we introduce the matrix C. 
so that 0^ = Co. For Z given above, and '^12^ we have 


33 


( 1 0 0 0 0 0 V 

0 0 0 1 0 0 I 

0 0 0 0 0 1 ' 


Letting a denote the element in the (r,s) position of Z. and Z 


denote the matrix 


dZ. 

1 

30 . 

irs 


we have: 


irs 


and 


9 log|z I 

— = tr(z7 Z. ) 

9c;. X irs 

irs 


3 Z ^ 

g- - - Z.^ Z. zT^ 

9 cr_.._ 1 irs 1 


irs 


Using these results we obtain 

V . 1 -1 


\ 1 


3o . 
irs 


m. tr(Z. Z. ) + trCZ~^ Z Z~^ ) 

2 1 ' 1 irs'^ 2 ^irs i i'' ’ 


and 


3^1og ^ 

30 ” Y ^i ^itu^i ^irs^ “ ^ 

itu irs ^ ^ ^ 1 irs i i' 


( 2 ) 


--^trCZ.^Z, Z.^Z, Z.^I ) . 

2 ' 1 irs i itu i i' * 

Recalling that E(M^) = m^Z^ we find that the negative expected value of 
the above expression is given by 
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^ m. tra.h. ) 

2 X X xrs X xtu'^ 


(3) 


Hence, the portion of the information matrix of corresponding to 
is the square matrix of dimension ^ p^Cp.+l) 'whose elements are .given by 

(3). 

In order to express (2) in a more tractable form, -we note that 

Pi 

S. = 2 a. 2. so that 

X , irs xrs 

r,s,“l 


Pi 


Pi 


tr(2,^2. ) = tr(2.^2. 2,^ 2 2. 2. ) = 2 tr(2.^2. 2.^2 )a 

X xrs^ ^ X xrs X ^ . itu itu'' ^ i irs i xtu^ itu. 

t,u=l t,u=l 


It can be confirmed that this is just the rs-th component of -W a,. 

i 

Similarly, the second term on the right side of (2) is just the rs-th com- 


ponent of the vector W (a.+h.)j where a, and h. are the vector forms of 

cr^ X X X X 

2^ and H^(ordered by columns). 

Hence, we have 


8 log 


3a. 

X 


3 log 


3a 




logI.j 


and since 


we obtain 




k 

2 

i=l 


(4) 


where W xs the total xnformatxon matrxx for a gxven by W = 2 C W C 

a , , X a. X 

X=1 X 


It has been tacitly assumed in the above development that all elements 
of p are estimable. This is not the case for all elements of a; hence, in 
(4) the vector a should be interpro.ted as the vector of estimable parameters 
of t. 
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In order to obtain estimates for y and a from (1) and (4) , we could 
use the method of steepest ascent to determine a stationary point for the 
likelihood function, since we have the expressions for the gradient vectors 

^ ^ and ^ An alternative is to equate (1) and (4) to zero, 

9y oo 

obtaining the likelihood equations 


W • y = Z D. W y. 

y i^l X PiX 


and W • cr = Z C: W (a.+hJ . (6) 

a . , X a. 1 i 

1=1 1 

Estimates for W ,W and h. can be obtained from initial estimates of 
y. a. 

the parameters- Then solving the likelihood equations for y and o by 

multiplying by and respectively, yields new estimates for y and a. 

Using these estimates to estimate W , W and h., the process can be 

^i i 

repeated. This process should converge rapidly, and at termination, we have 

“ 1-1 ~ ~ 
estimates of and ,’ the large sample covariance matrices of y and a, 

respectively. 


Consider the case k=2, that is, we have a set of m^^ complete observa- 
tion vectors and a set of m^ incomplete vectors. The large sample covariance 

-1 ■ T -1 

matrix is W where W = W + C„W C„ . Hence W can. be written as 
a 0 2 2 a 


Tiie second term represents the gain in precision which may be expected 
if both groups of data arc used as opposed to only the complete observation 
vectors to estimate Z. 
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Considering the variances of each component of a we obtain the following: 

var(o..) = 2 a../N if the i-th component of the data vectors is not 
11 11 

missing in 
^2 A 2 

and var(a. ,) = (1- rr— R.) 2 a . , /n, otherwise. 

11 N 1 11 

Although the likelihood equations (5), (6) in general must be solved 

numerically as indicated, there are certain special cases in which they may 

be solved analytically. This is true for any situation Xv'ith only two groups 

of data where one group consists of the complete observation vectors, and 

for nested incomplete vector observations. By nested we mean that group 

consists of mj^>p complete vectors and it is possible to label the remain- 

ing groups so that is a principal submatrix of for i=l,2, . . , ,k-l. 

Consider the likelihood equations (5), (6) for the nested case. They 

can be solved sequentially as follows: Consider equations' (5) and (6) with 

k=2 and replace the elements of W and W in (5) by their estimates using 

^i ^2 . 

only. The solution of (5) yields \i which is maximum likelihood for y if 
k=2. Substituting y for. y in h. and using a, to estimate and W" in (6) 

. t °1 ^2 

enables us to solve for o, say cr, which is maximum likelihood for a if k=2. 

We next consider k=3 and repeat the above process with a as initial estimate 
rather than The resulting estimates are maximum likelihood for k=3. 

This is' continued until all k groups are exhausted and the final solutions 
are maximum likelihood. 


For the remote sensing application, it is expected that k“2 will be the 
most frequently occurring situation since a portion of the subvectors repre- 
senting data for a particular pass at a point in time over a certain region 
will be missing due to partial cloud cover. 
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5 . Derivation of Maximum Likelihood Classifier 

We now consider the problem of classifying an observation vector in one 
of two multivariate normal populations when dealing with incomplete data 
vectors. It will be assumed that sufficient training data are available to 
estimate the covariance matrices very accurately. Hence, we assume the 
covariance matrices are known and unequal. These will be denoted by and 
2 ^ with corresponding submatrices and . Let denote an observation 
vector to be classified into either or N(p 25 S 2 ). Letting denote 

the samples from and from logarithm of the likeli- 

hood function L^ of all the observations can be expressed as 


logL^ = logK^ - — 


n. 

J 


k ( 


+ (Xo-v.q)'' III 

vhere Kj =• n (2rt ^ ^|Zy| ^ ^ . aad 2.^ Is 

3 


the covariance matrix corresponding to X^ of size p_. Likewise, u.„ is the 

U G xO 

mean vector corresponding to X^ for 1=1,2. Note that 

k . T -1 

log = log K. + 2^ (log Ly + log Ly) + z'J 


Hence, 


8 log Lj 


= Z 


T 3 log 
d: — - — 
j-i ^ ’“ij 


+ 1 


-1 

10 


'’'o - 


>'io> 
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8 log L . j 

ii. =: T y 

a-1 « 




is to be multiplied on the left by the appropriate; D, if Pq'^P* This will 

T —1 

be denoted by writing ^10^^0~^10^* have 


3 log L- k 


1 T -1— T -1 T -1 T 

T — -= E m.D:(E,:x.) + d,\.e,;!;x- - e m.CDTE.tD.) + d::.e,;Ip,_ y, . 
8Pj^ 3 J Ij J 10 10 0 ■ ._2 1 J Ij 3 10 10 10 1 


Similarly , 

3 log L ■ k k - 

— r- E n.DTE.tY. - E n.DtE.fl). y_ 

8^2 j=l 3 3 2-2 3 1 J 23 ^ 2 

For ease of presentation we introduce the folloxTing definitions: 


Q„ - E mJD^zrV) + q„ = Z (Z,^XJ + 


■11 3"lj l' "10"10"10’ 


j_2 3 3 Ij J 10 10 0 


T -L 

- E m.D.E-,7D. 
12 3 3 I3 3 


^12 hi.dP ?3C. 
3==1 3 3 I3 3 


^22 ^20^20^20’ 
3 


’22 = 4,-"3”^2K + 


Q„., = E u.D^e"^. 
21 3 3 2j 3 


T -Ir 

I,., «= E n.DTE.tsr. 
21 . , 3 3 22 2 


S et ting 


3 log'L., 


3 log L-, 


■’^^7 °> “ 3 ^;^ “ '*ll'‘l “ %1 ‘^2l'‘2 ■ ■>2r 


Similarly 


3 log L., 


3 log L2 

= 0 yields 93^2’^! "" '^12 ^22^2 "" ^22 


Thus, the maximum likelihood estimates for P2»P2 hypothesis 

T T 

that Xq is from N(Dj^q 1 j^, O^qE^D^^) is given by: 
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1^1 ^ 11 ^ 11 » 1^2 ~ ^ 21^21 * 

Likewise, if is from NCD^q 2 > ^20^2^20^’ maximum likelihood 


estimates for 


2^1 ^ 12^^12 2^2 ^ 22^22 * 


Letting L^ denote the maximum of L^(i=l,2) under variation of Wj^,p 2 
we get 

k 


n. 


P.(m,+n.) 

- J -T -T _®_ __i _1 _10 

^ = n (2ir) 2 2jjj.^j 2 2^2,r) ^ 


3=1 


2j 


10'. 


r m. 


exp I - - _Z^ 


n, 

3 




(’'jcc- l“ 23 > 


L„ = 


k 

n (27 t) 

3=1 


P. (m.+n.) 
_ -3-- -3 3 

2 


^^0 ’^ lO ^ ^10 ^^0 ^ 10 ^ 


n. 




exp 


“j 

2 -2^U>'^U»3aVu) + 


1 

- ■= z 


<"3.-2‘'2j> 




Forming -2 log and simplifying, we get 

T 

"2 
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I ^10 1 ^ T -1 '' ± “I 

-2 log |Z I '*' ^10^^0”l^l0^ ~ ^^0“2^20^’ ^20^^0~2^20^ 

L 2 20 ■ 

*^12^12^12 " ^21^2l‘^21 “ ^11^11^^0^10^'^ll‘^H ‘*22^22^^20 ^20^20^ ^22^22 


^ 11 *^ 11^11 ■*■ ^‘^ 11 ^ 11 ^ 10^10 ^0 ^ 22 ^ 22^22 " ^‘^ 22 ^ 22 ^ 20^20 ^0 


size Pq<P. 


If L(Z) = -2 log ^ = L(xJ,X 2 ,...,X^,Xq,yJ,...,Y^), then classify X^ 
^2 

in N(vi^,2^) if L(Z)<0 and in N(y2>22) if L(X)>0. 

The above expression simplifies whenever is. pxl. In that case, 


L ! ^ f 

-2 log ^ = 1„ 1 ^0 + 'jL'^lka •iW'>22 ~ ’2i‘’S'i2i 
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PROBABILITY OF MISCLASSIFICATION WITH MISSING -DATA 
1 . Introduction 

In the problem of discriminant analysis the probability of misclassifica- 
tion is of significant importance. The difficulty in obtaining explicit 
expressions for the measure of probability of misclassification has prompted 
various developments of statistical methodology of classification. The 
problem of misclassification is further complicated by the problem of missing 
data. This report investigates the relationship between the probability of 
missing data and the probability of misclassification. Since divergence is 
a widely used measure of distance, it will be Investigated as a criteria 
upon which to base a discriminant function. 


2. Definition of the Problem 


Let X = 


^1 

nxi 

^2 

nxl 




MVN (]i, 
np ’ 


£ ) 

npxnp 


X 

P ; 

nxl ; 
npxl 


( 1 ) 


where X consists of p subvectors, each of length n. The data will be lost 
in such a way that an entire subvector will be lost. Let be the probability 
that the 1 subvector is missing, i=l,2,...,p. 
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As a mechanism for identifying the outcome of missing data define I as 

a random vector of o's and I’s, with 0 indicating a missing subvector. I 

will be of length p, i.e. for some j, I"; = (Oil ... 1) indicates the 

J 

first subvector, X , is missing. That is, we use the notation I to denote 
^ ■ d . 

a value of I. There will be 2^ possible outcomes of I which are mutually 
exclusive and independent. The probability of outcome I above can be 

j 

calculated: 

P[IJ = (0 1 1 1 ... 1 )] = q^Cl-q^) (l-q 3 ) . . . ' (l-q^) = y.. (2) 

Let ir-, ir^, ... ir be the m populations and p^ , p_, . p be the 

coirrespoiiding a priori probabilities. Let P[i|j] be the probability of 
classifying an observation from the population into the i^^ population 
and C(ijj) = the associated cost. For the remainder of this paper the cost 
C(i|j) = 1 if i j and C(iji) = 0. 

If f^(x) represents the density function of the i^ population, 
divergence . A, is defined by 

CO 

A - / [f^(x) - f (x)] An [f^(x)/f (x)3 dx . (3) 

Letting f^(x), 1=1, 2,..., m be the multivariate normal density function 
with mean vector and variance-covariance matrix S^', i=l,2, ...m, we get 
divergence 

A = I {tr(Z -L ) + (z:Vrt (p,-y.) (p.-p.) ’} (4) 

^ J- I JX xj XJXJ 

3. Equal Variance-Covariance Matrix Case 

Anderson [1] considers the case for equal variance-covariance matrices 
and multivariate normal distribution. Divergence then reduces to 


( 5 ) 
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In the case of two populations and a priori probabilities and p^, 
the probability of drawing a sample from population ^2 and misclassifylng it 
into 7T^ is P( 1 | 2 )p 2 and the probability of, drawing a sample from population 
7T^ and misclassifylng it into is P(2|l)p^. Thus the total probability of 
misclassification is 

P[misclassificati 9 nj' = P 2 P(l|2) + p^ P(2|l) (6) 

Anderson shows the discriminant function which minimizes the P[mlsclassi— 


fication] is the usual Bayes discriminant function: 

The best regions of classification are given by, classify into if 

( 1 )^( 2 )^ , ^- 1 ^.^( 1 )_^( 2 )^ ^ ^ 

Otherwise classify into ir^. 

Anderson finds the distribution function for (7) and using this distribu- 
tion obtains the probability of misclassification as follows (still with 


^1 ^2 • 


P(l[2) = / 77 — 

iin P2/P2 + 


1 2 


dy 


and 


^^P2/Pl 



1 2 

- 

e dy . 


(9) 


( 10 ) 


Both probabilities are monotonically decreasing functions of A. Therefore, 
the total probability of misclassification is given by: 

P [mis classification] = P 2 P(l|2) + p^ P(2|l) ( 11 ) 

which is a monotonically decreasing function of divergence. 



4. The Bayes Discriminant Function with Missing Data 

We wish to incorporate the probability of missing .subvectors into the 
problem of the probability of misclassification. 

For the remainder, of this report we will for convenience restrict 
ourselves to the case of two populations, -rr^ and tt 2 > with a priori 
probabilities p^ and p^. The extension to n populations is obvious. We 
will consider Z = Z^ =2 and take n=4 and p=4. Of course the development 
which follows is the same for any n and p but we have selected these values 
from the physical model for definiteness. Thus 


'X' MVN ( p , Z ) (12) 

16x1 16x16 

16x1' 

4 

There are 2 =16 ways in which the subvectors can be lost and 16 

mutually exclusive and independent outcomes of I with associated probabilities 
Y^j j=l»2,... 16. For example, if = (1,0, 1,0) we have 

= (1,0, 1,0)] = (l-q^) ^2 *14 

16 

(Clearly E y “ D* 

j=l J ■ 



Definition 1: 


Define D(U,I^) as the usual Bayes discrimination function, but based 
only on the available data as reflected in outcome Ij . 

In calculating the probability of misclassification using D(U,Ij) the 
probability of misclassification is based on the parameter 




( 14 ) 


where the subscript indicates that the parameter is caluclated only from the 
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subsets of y and Z associated with the available data in outcome I.. Only 
in the outcome I 2 “ ^1’ 1> 1» 1) > where no data is missing does = A, 

The distribution of the Bayes discrimination function has been altered 


by including the probability of missing data. It is now the function of 
two random variables. Thus, the probabilities of misclassification and 
outcome I, are given by: 


P[li2,Ij]P[I^] 


CO 


/ 

T) . 

Pj/Pi 




1 2 


e 


dy 




( 15 ) 


P[2[l,I^]P[IjJ 


an P2/P2 

/ 



1 

/zir 



T. 

J 




(16) 


The total probability of misclassification and outcome I, 


P [misclassification and I.] = 

3 


^2 ^ 


1 2 


ve 

P2^Pl + 


is : 


dy 


+ p 


1 


f ^ 



1 

/2tt 


1 2 


dy 


(17) 


We can then define an "expected probability of misclassification, " 
which is actually just the probability of misclassification in the problem 
as we have now modeled it. We have simply used the term "expected 
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misclassification" to emphasize that we have accounted for the possibility of 
missing data in our calculations, thus 


P [misclassification] = E^. P [misclassification] 


16 

= 2 y 

3=1 ^ 


^2 ^ 


1 2 


An p2^Pl ~ 


a , 


a . 


.to pj/p^ + 


— a dy + p / 

v27T -o 


1 2 
1 ■ F 

— e ^ dy 


(18) 


Figures A-1 through A-15 demonstrate the behavior of (18) with 
different y^, i.e. different 'll “ ^2 ^ *^3 


Definition 2 ; 

In the case where all the data is missing but a region must be classified, 
we can classify the region by making use of the a priori probabilities . That 
is, we simply sample from a random device with those a priori probabilities 
and make the classification. For example, in the two population case we take 
a "coin" which has probability p^^ of heads and p 2 = ^"^i tails. Then we 
flip the coin and classify the region accordingly. Thus we have once more 
P [misclassification] = P[l|2] p^ + P[2jl] p^ 

= ,p^p2 + P 2 P 1 = 2p^P2 (19) 

= 2p^(l-p^) 

Thus, if we denote outcome (0, 0, 0, 0) by we have 



P [misclassification] 


15 

2 y [P2P(1|2) .+ p^ P(l|2)] + 2y^g Pid-p^) 

j=l ^ 


15 

= 2 y.[P2P(l|2) + P3^P(2|1)] + 2 qiq2q3q4 

j=l 


)) 


One should note that if p 2 ^=’p 2 (^0) is equivalent to taking = 0 which 


is certainly reasonable. 
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By definition (and by a realistic approach to the physical problem) 

the data obtained will be considered "essential data" in the sense that any 

data lost will increase the probability of misclassif ication, i.e. 

P[ilj, > Ptih*. I^] (21) 

if I represents an outcome where more data is lost than in outcome I . i.e. 
k 

every element in I is contained in I . Since the outcome of I determines 
k ^ 

the parameter in the calculation of the probability of misclassification, 
this imposes the condition in (21) above that This makes the 

definition more suitable mathematically, so we will make this our formal 
definition. 


Definition 3. Data will be said to be essential data if every element of I, 

iC 

is in I. implies ct, < a„. 

a ^ k * £ 

One such case would be where 2“^ =-diag [-i- ] i = 1,2,.., 16. 

'^il 

Theorem 1 

Let q = (q^,q 2 ,q 3 ,q^) and consider two cases. Case 1: at least one 

of the q^ is greater than 0. Case 2: q^=0, Vi. This implies ’no missing 

data, i.e. the only outcome possible for I is = (1»1>1»1)., Then 

E[P[misclassif ication; I.] in case 1] ^ E[P [misclassification; I.] in case 2] 

3 3 

Proof ; 

Since Case 2 is trivially true we consider Case 1 where we let 
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Now in case 1, if there exists at least one greater than zero, 
becomes 0 < y-. < 1 and there exists at least one other y. greater than 0. 

X 3 

Hence 


Ej[P(ir2)] = y^ / 


e dy + 


tn + 4 


+ / 


e dy + 


1 "F 


itn P^/Pi+V 


1 fi 

in p^/p^ + — 


But by definition 3, each < A and thus each integral of equation (23) 

(other than the first integral) is greater than C and equation (23) can be 


Ej[P(lj 2) ] > y^C + y2C + ,.,+yjC + ... + 


2 y.C 
i=l ^ 


since E V. = 1 

* !L 

. i=l 


xn a sxmilar fashion, we can show 
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Ej[P(2|l) for at least one > 0] > E[P(2il, 1^) q^- = 0,Y’l] , 
and finally 

P [misclassification; if at least one q^ > 0] > P [misclassification; 

q. = 0, V q. ] 

5. P [misclassification] as a function of divergence 

Unfortunately, P [misclassification] = [P [misclassification] ] is not 
in general a decreasing function of divergence, as shown in the following 
example . 

Example: Let the divergence between populations -n yit ^ be the following 

(without loss of generality we can assume equal a priori probabilities) . 

Let A^(ir^,Tr 2 ,S^ - ^ 2 ) = 16 A2(’’’2>^3>^2~^3^ ~ in .the following way; 



16 X 16 16 X 1 


= 4(3.99) + 1K.003) + K.007) = 16 
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16 X 16 16x1 

= 4(2.75) + 11(2) + 1(3) = 36 . 


Now let = .5 and q 2 ~ ~ ^4 ® * 

Then = *5 and - ,5 and all the remaining terms, y^ = 0. The only two 

outcomes of I that are possible are = (1111) and = (0111). 

Thus 

E[P(1|2,A^ = .5 (.0228) + .5 (.“4602) ^ .2415 

and 

E[P(1|2,A2) = .5(.0013) + .5(.0065) = .0039 . 

So, even though A 2 

E[P(1|2,A2)I <E[P(lj2,A3^)]. 

6. Equal q^, i=l,2,3,4 

While it is probably true that for equal E and q = 93^“92~‘^3~‘^4’ 

P [misclassification] is an increasing function of q, it may be hard to show 
analytically . 



130 


Consider the two dimensional case and equal a priori as a function of 
“ P [mis classification] 

= (1-q)^ + q(l-q) + q(l-q) + q^C^ 0 £ q < 1 

where by def. 3 ^2*^3 ^ ^4 ' 

Now g(0) = and g(l) == so we know 0 < g(0) < g(l) . Note that this is 
true for any dimensional case. 

However, in general to prove there are no critical values for n dimensions 
would involve proving there are no roots for an n-1 degree polynomial for 
0 _< q ^ 1. This may be very difficult to show analytically. 

The question also arises with equal q^ whether or not P [mis classification] 
= Ej[P [misclassification] ] is a monotonically decreasing function of divergence 
in the equal covariance case, even though it has been shown not to be true 
for unequal q. Again this is probably true but would be difficult to prove 
analytically. 

7. Unequal Covariance Matrices Case 

It would be desirable to extend the above theory to encompass the case 
of unequal covariance matrices. However, Chang[4] points out that divergence 
is neither uniquely nor monotonically related to Bayes ' classification errors . 
Since the relationship of the new model which incorporates the probability of 
missing data and divergence with equal covariance matrices is still uncertain, 
further investigation into the properties and possibly other discriminant 
functions should be investigated. 
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8. Plots 

Appendix A contains several plots of probabilities of misclassification 
for equal q^*s and S = I under different a priori assumptions. 
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Appendix A. Plots- of Probability of Misclassification 

Figures A-1 through A~15 contain plots of the probability of misclassifi- 
caion. P(1 GIVEN 2) denotes P(l|2) and is given by 

16 

P(l|2) - E- Y [1 - 0>(A )]■ 

- where (1) $ = standard normal distribution function 
(2) A^ = Jin P2/P1 + 


P(2 GIVEN 1) denotes P(2ll) and is given bj 

16 

P(2[l) = E T. ^(BJ 
I ^ 


where B. = An p^/p, 
J ^2 1 



The total probability of misclassification is 

P [misclassification] = p^ P(l|2) + P-, P(2|i) . 

For each plot E = I. Let 

dj^ = i^^ component of 

The values for the d^'s for each of the plots is given in TableA.l-. 
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1.0 Introduction 

Much work has been done in recent years concerning the problem 
of extrapolating a set of irregularly spaced data to obtain a surface 
(possibly continuous but not necessarily) covering a large region of 
interest. The example used in this paper to illustrate the various 
extrapolation procedures will be an agricultural one but the methods 
can be used in other fields such as biology, meteorology, and city 
planning. 

Suppose that we are given yield data (bushels per acre) for a 
particular agricultural crop such as wheat at several irregularly 
spaced locations within a region R. Designate these data as 
(xi,yi,Z 2 ), ... , where Zj_ = the yield at point (Xj^,y^l^) : 

See Figure 1. 



X 
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Consider these yield data as being particular points on a 
surface, henceforth referred to as a yield surface . The problem 
then is to devise a method which extrapolates this set of data 
over the entire region R, producing a yield estimate for each 
point of R, The term "extrapolate" as used in this paper may 
actually mean "interpolate" if the point under consideration lies 
within the convex hull defined by the data points. All of the 
proposed procedures for solving this problem will assume that a 
grid (coordinate system) has been constructed over R with each 
data point assigned to the nearest grid point. The problem then 
reduces to extrapolating the input data to each grid point lying 
within R. 

A number of techniques {[l],[4],[8l) exist for interpolating 
within the convex hull of a set of data points, where the convex 
hull is the convex region of minimum area containing all of the 
data. The methods examined in this paper were selected because 
of their ability to extrapolate outside the convex hull of a sparse 
data set. One method (referred to as Method 3) is to fit the data 
points with a least squares linear surface of the form 
2i = Cq + c^xi + C2yi j i = 1,2,...,n. 

McLain [ 3 ] expands this approach to . include quadratic terms and 
(more importantly) uses a weighted least squares regression to 
determine the Cj^'s where the weights reflect the distance of the 
data points from (x^^,y^) . Consequently, a new set of regression 
coefficients is derived for each extrapolated point on the yield 
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surface. This method is referred to as Method 10 and will be 
discussed further in Section 2.9. 

Shepard [ 9 ] has derived a surface generation technique which 
produces a surface that is continuous and passes, through the data 
points. The method uses a weighted average of the data points to 
estimate the surface height at a given point where the weights 
reflect the relative distance and direction of the data points . 

This method is referred to as Method 9 in this paper and will be 
described in more detail in Section 2.8. 

The methods described in subsequent sections were chosen 
because they seemed to be likely candidates for extrapolating 
sparse data. Because of the assumption of sparse data, the degree 
of sophistication the extrapolating procedure should have is 
questionable . One could argue that with an extremely sparse data 
set of yield data that using the sample mean to estimate the 
entire yield surface would be a "safe and reasonable" thing to do. 

On the other hand, one should possibly use a more sophisticated 
technique to glean as much information as possible from the set 
of data, although it becomes increasingly difficult to substantiate 
this sophistication from your sample data. Consequently, the authors 
present and compare ten methods ranging from the sample mean to 
the highly sophisticated procedure proposed by Shepard. It is 
hoped that the reader will apply each of these procedures to his 
own particular application to determine which of them most accurately 


models his situation. 
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In Section 3.0, the authors compare these methods using five 
years of wheat yield data from North Dakota. For this study, five 
years of data existed for seven yield test stations along with 
yield data for approximately 45 check points (cities) . A Fortran 
contour mapping program was written and used on several of the 
methods to compare the final yield surfaces to each other and to 
a contour map of the full set of available data (i.e. , the 52 
yield data points) . The latter map was assumed to accurately 
model the actual wheat yield distribution of North Dakota and 
was used as a check of the various extrapolation procedures. 

Section 4.0 contains the conclusions of this study along with 
a table containing the desirable and undesirable characteristics 
of each model. 

2 . 0 Proposed Methods of Solution 

As mentioned previously, these methods will vary considerably 
in their complexity beginning with a very simple technique of 
yield extrapolation (Method 1) to the more sophisticated procedures 
of Shepard and McLain (Methods 9 and 10) . 



Figure 2 
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2.1 Method 1 - Composite Average 

If one wishes to ignore the relative location of the yield 
data points, then a reasonable estimator for the yield at any 
point in R would be the sample mean, i.e. 

2p 1 = yield estimate at P using Method 1 
' 1 N 

■ ^ 1 'i ^ " 

where = yield at P^. 

This method would be appropriate if the user ignores location 
because of the extreme spare ity of the data or he feels that the 
yield values are randomly distributed about R. This estimator 
provides a continuous surface which is well behaved near the 
boundary of R. However, it does not reflect any variations within 
the region, in particular in small neighborhoods about each data 
One rationale behind using z is that while it may serve 
as a relatively poor local (point) estimator it should provide 
an accurate global estimate of the total yield for the region, R. 

2.2 Method 2 - Nearest Neighbor (NN) 

This method (another conservative estimator) estimates the 
yield at P by the yield of that data point which is nearest P, i.e. 

Zp ^2 “ yield estimate at P using Method 2 

= Zj^, 

where (1) k is such that d^ = Min {d- 

1 £ j ^ N 

(2) dj = Euclidean distance from P to P j . 

This estimator provides better estimates in small neighborhoods 
about each data point and also is well behaved near the boundaries 
of R. However, the resulting yield surface is discontinuous with 
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the maximum yield estimate being the maximum z^. For each point 
P, this estimator completely ignores the yield information of the 


N-1 furthest data points. 

2.3 Method 3 - Least Squares Linear Surface (LS) 

The Method 3 estimator fits the yield data with a plane that 
provides a least squares fit to the set of yield data. So 

Zp^3 = Cq + c^Xp + C2yp 

where (1) coordinates of P are (xp,yp) 

(2) Co,Cj^,C 2 are given by 

‘^l “ 5 ^®yy®xz “ ®xy®yz^ 

*^2 ~ D ^®xx®yz “ ®xy^xz^ 


and 




(3) Sjjjj = 2 (x^-x) 

^ - 2 
Syy = 2 (y.-y) 

K _ _ 

®xy = ^ (y^-y) 

N __ 

®XZ “ ^ (x.“X)Z 

ill 

■ N _ 

= 2 (y^-y) 2^ 


yz ^ 
N 


N 


s =EZ:j,s =Sy.,s 

z 1 ^ y 1 ^ ^ 


N 

2 X • 
1 


^ ®xx®yy ®xy®xy 

This estimator provides intuitively "good" yield estimates in 
the convex hull (see Section 2.4) of the data points. The resulting 
surface is continuous and does reflect trends -in the data (e.g. the 
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yield values increase from North to South) . The major problem 
with this technique, as with any linear regression technique, is 
that it may be extremely ill behaved outside the range of the input 
data points, i.e. near the boundary of R. . If the yield surface 
(a plane) is extremely inclined, one can encounter negative or 
unrealistically large yield estimates. 

2.4 Method 4 - LS/NN 

This method attempts to provide more conservative yield 
estimates for points outside the convex hull defined by the input • 
points. The convex hull of a set of data points is defined to be 
that convex region of minimum area containing all of the points. 

The authors have developed an algorithm (see Appendix A) and 
computer program (Fortran) to determine the convex hull of an 
input set of two-dimensional data points. Figure 3 illustrates 
this for a set of six data- points. 



Figure 3 

The LS/NN estimator uses the LS estimator (Method 3) inside 
the convex hull and the nearest neighbor estimator (Method 2) 
outside this region, i.e. 

Zp ^ “|^P ,3 if P e convex hull of data points 



otherwise 
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The resulting surface although possibly better behaved near the 

boundary of R will be discontinuous at the boundary of the convex 

hull. However, for data that varies considerably, this more 

conservative estimator may be much more desirable than the overall 

regression estimator. 

2.5 Method 5,6 - Average Linkage 

The average linkage estimator predicts yield by a using a 

weighted average of the data points. Thus 
^ N N 

Zp 4 = {2 g(di)z )/(i: g(d.)- ) 

' i=l • ^ i=l 

where (1) d^ = distance from P to P. 

^ 1 

(2) g(d. ) is a monotonically decreasing function of d. such 
^ 1 

that g(d^)— ^ o as d^ — 

For j=5, g(d. ) = 1/d- and for j=6, g(d.) = 1/d-^. The Method 6 

X XI 

estimator will reduce the effect of distant data points more so 
than Method 5, i.e. the effect of the i-th data point on the 
yield estimate at P "dies out" faster using Method 6. If one 
wishes to further decrease the -effect of distant data points, a 
possible candidate would' be g(dj_) = e or e ^ . 

The resulting yield surface using either method will be 
continuous and is well behaved near the boundary of R. Moreover, 
it includes the effect of each data point yet is completely 
dominated by the yield at P^^ for any P in a small neighborhood 
about P^. The resulting surface is conservative, however, since 
the maximum yield value will occur at one of the data points. The 
average linkage method is also unable to reflect any directional 
trends in the data. 
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2.6 Method 7 - Average Linkage v/ith Directional Correlation 

The Method 7 estimator attempts to improve the average linkage 
estimator by incorporating significant directional* trends that are 
present in the data. 


where (1) 


w. 


(2) w, = 


'X 


6 + 

Wi^p 

,7,x + ^2^P,7,y 

y>' 

if 



if 

6^ = 5y = 0 

V’ 

if 

or 6y = 1 


if 

= S'" 

= 1 + 

Si 

Py,4 1 =5 l«x 


“1 


(4) 6^ = 




1 if |px,z|-- P: 


0 


min 


if ^min 


1 if Pm-i 


min 


0 if Pmin 


(6) p „ = sample correlation between x.'s and z *s 

(7) p = sample correlation between y^^’s and 

y / z 

( 3)2 = yield estimate at P obtained by regressing yield 

P,7,x 

on the x-coordinate , i.e. = •^x' 

for regression coefficients 

: yield estimate at P obtained by regressing yield 
on the y-coordinate, i.e. ^y ®y * 

for regression coefficients ^y'^y* 


(9) z 


P,7 ,y 


(10) w » P are predetermined constants, 

max roin 
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'Vax measures the maximum importance that the 

experimenter wishes to give to the directional trend estimates 
Pjjjin minimum significant directional correlation that 



Here we have yield sample data of 40,20, and 70 at P ,P.p,P 

1^3 

respectively. Setting p . = .8, and computing directional 

min 

correlations, then S ^ = .114 (not significant) and p„ = .992 

X f Z y f Z 

(significant) . Regressing yield on the y coordinates produces 

2p 7 = 13.78 + 2.86 y . Letting w = .8, then w = .8 x .992 

^ r ' rY P max 

= .794, and K n ^ -206 c- + .794 ^ 

P^7 P,6 P/7,y 


Thus; 


p g + .794 X (13.78 + 2.86 y^) 


tn ,7 


— .206 z 

= .206 (49.36) + 10.941 + 2.271«(19) 

= 64.26 


z^ 7 = .206 (61.61) + 10.941 + 2.271-(29) 
2 '^ 

= 89.49 

z = .206 (40.00) + 10.941 + 2.271«(8) 

/ / 


37.35 
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^ ^ -206 (42.08) + 10.941 + 2.271 (11) 

1^4/ ' 

= 44.59 

The choice of w is, of course, very subjective. If the 
in 3 .!^ 

experimenter feels that apparent directional data trends should be 
heavily considered, then a large value of would be appropriate. 

If one is uncertain about such trends, then a lesser value of 
^max should be used, and if one wishes to totally disregard 

any apparent trends, then the average linkage estimator should be 
used. It should be pointed out that ^ (the average linkage 
estimator) can be replaced by any of the remaining methods in this 
paper. Whichever method one uses in place of z the result of 

F 7 b 

Zp ^7 wxll be to incorporate the effect of directional correlations 
into this yield estimator. Another question to consider in choosing 
'^max "how representative are the data points of the entire 
region, R?" If the points are, say, clustered in a small portion 
of R, then one may be wary of directional predictors and may wish 
to use a more conservative estimator by choosing a small value for 
'^max. points are distributed evenly about the 

entire region then this would suggest using a larger value for 
w 

max 

The resulting yield surface will be continuous, but could, in 
some cases, be ill behaved near the boundary of R. This can be 
controlled somewhat by the choice of w 
2.7 Method 8 - Objective Analysis 

This method is a result of a meteorological approach to 
sparse data extrapolation. Sasaki ([5], [ 6 ], [7]) and Wagner [-10] 
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have proposed such a method referred to as an objective approach 
to analyzing data fields. This method also utilizes a low pass 
filter to compensate for the spar city of data and to suppress the 
high-frequency noise contained in the initial data. This procedure 
can be broken down into the three following steps: 

Step 1 : Initialization 

The region of interest, R, is fitted with a grid and, as in 
all the previous methods, the object will be to provide a yield 
estimate at each such grid point. The initialization process 
begins by assigning the N sample values to their nearest grid point 
and setting the remaining z (yield estimate) values equal to zero. 
Finally, R is augmented by a set of boundary grid values, all set 
equal to zero (see Figure 4) . Call the new region R . 



Figure 5 
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Referring to Figure 5, consider an arbitrary grid point P. 

The initialization process is actually a seguential procedure which 

continues until all grid points have been assigned a non-negative 

yield estimate. At each step, the yield estimate at point P is z^, 

where • ^I^'xisting if Zp > 0 

average of the if nt least one of 

9 points in Figure 5 these points is > ' 

unspecified if all 9 points = i 

Thus on the first pass, z = the existing z only if point P is 

p 

one of the original N data points. After point Pis finally 
assigned a non-zero initial estimate, it remains unchanged on 
succeeding steps of the initialization procedure. This procedure 
continues until ^ of the grid points have been assigned on non- 
zero initial yield estimate (i.e. no Zp is unspecified) . 

Step 2: Smoothing 

As a first attempt to filter out some of the high— frequency 
components of the initialization process, a smoothing process 
(filter) is applied. 



Figure 

Considering, for the moment, a one dimensional sxtuation, 
thrs smoothing filter can be written as 

■ (1 - S) + I + f j-i) 


(2.7.1) 
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■for an arbitrary function f^ and constant S.‘ For S = 1/2, (2.7.1) 
reduces to = 1/4 (f + 2f ^ + f^_^) • 

Referring to Figure 6 for the two-dimensional case, the 
smoothing algorithm replaces Zp by its "smoothed" value 

Zp = ^ W-Z./16 (2.7.2) 

P 1=1 1 2 .' 

where (1) the index runs over the 9 points in Figure 6 

(2) the weights (w^) are the numbers to the left of each 
point in Figure 6 . 

For points on the boundary (but not the corner) of R (not 
R"'), the nine point weighted average of (2.7.2) is replaced by 
the corresponding six point average where the weights are illustrated 
in Figure 7. Finally, using' Figure 8, the smoothed values for the 
corner points of R can be determined using a four point weighted 
average. Note that for this case, the denominator of (2.7.2) is 
replaced by 8.0. 



Figure 7 



Step 3: Extrapolation Using Observational and Low Pass 
Filtering Constraints 

Briefly, this step attempts to define a surface (field) of 
data which does not differ a great deal from the surface generated 
by Step 2 yet, at the same time, contains a minimum of high- 
frequency modes. The problem is essentially one belonging to the 
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Calculus of Variations where the purpose is to minimize I, given 
in (2.7.3). 

I = [a(z - z')^ + (2.7.3) 

where (1) z represents the final surface generated by Step 2 

(2) z is the desired yield estimate after Step 3 

(3) V^z is a difference operator applied to the yield 
®sti'^stes which controls their rate of change across the^ grid 
points of R 

(4) a and are predetermined weighting factors. 

Thus Step 3 determines a yield surface which resembles . that 
of Step 2 but minimizes rapid changes in yield across the grid 
points of R. The Method 8 estimator is the outcome of Step 3 
with' the addition of another term in the summation in (2.7.3), 
namely another low pass filter term of higher order'. 

I" = [a(z-z")^ + (Vz)^ + (V^z)^] (2.7.4) 

For a more in depth and concise description of Sasaki's approach 
and difference operators, see Haltiner [2, Chapter 14.5] 1 

The Step 3 algorithm proposed by Wagner [10], minimizes 1' 
by determining the Euler-Lagrange .equations corresponding to (2.7.4) 
and then solving these equations by a numerical method known as 
relaxation . This method is an iterative solution which continues 
until the yield at each grid point of R changes by an amount less 
than some predetermined amount. It is similar to the procedures 
of Steps 1 and 2 in that a weighted average (using a, and a 2 ) 

is computed for each grid point of R and compared to the existing 
yield value. The yield estimate at the grid point is then adjusted 
towards (not see equal to) this new estimate and the procedure 
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continues to the next grid point. 

The choice of weighting factors in (2.7.4) is of course 
very subjective. In Wagner's thesis, he often uses weights of 
a = 100, a^= = 1, i.e. placing a very high weight on fitting 

the Step 2 surface (referred to as the "observed" values by 
Wagner) . 

In summary, the resulting surface using Method 8 will be a 
"smooth" surface which will be well behaved throughout R but will 
not exactly fit the original set of N data points. For a set of 
data of low variance and using a large a value (e.g. a = 100) 
we would expect the final yield surface to strongly resemble the 
"smoothed" surface generated by Step 2. For this reason, there 
appears to be a serious lack of attention paid to this smoothing 
procedure and, in fact, Wagner essentially describes his low-pass 
filtering approach beginning with a complete grid of smoothed data. 
2 . 8 Method 9 - Modified Linkage 

This approach, proposed by Shepard [ 9], is an extension of 
the average linkage estimators (Methods 5,6).- Shepard attempts to 
improve the straight weighted average estimator by (1) selecting 
only nearby data points to be used in estimating the yield at 
some grid point in R, (2) including the effect of the directions 
between this grid point and the data points used in the estimation, 
and (3) correcting for the zero gradients at the N data points on 
the yield surface generated by Methods 5 and 6 . 

To correct for the zero gradient at data point P^, small 
increments were added to nearby data points so that the generated 
yield surface would have "desired" partial deviatives at P^. Since 
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we are assuming a sparse data set (e.g. 1% grid occupancy) the authors 
did not feel that this modification would improve the accuracy of the 
linkage estimators. Later experiments showed this to be true. 
Consequently , only the nearby point rule and the direction correction 
were used in the Method 9 estimator. 

Step 1: Selecting Nearby Points 

Z ^±z- 

Denote the Method 6 estimator by f^(P) = 1 

Z w. 

2 ^ 
where w^ = 1/d^ , i = 1,2,...,N. 

Since this estimator practically removes the effect of distant 

data points and we are using a sparse data set, using only nearby 

data values may not provide a significant improvement in estimation. 

The resulting suface should however contain less noise.. 

The first step is to define a circle of radius R, which when 

constructed arbritrarily about R, will contain ( on an average) 

a predetermined number, n^, of data points. Thus 

Trr^ = nj^« (A/N) 

where A = approximate total area of R. The choice of n, will 

primarily be a function of the sparcity of the data. For each grid 

point P, a collection of data points near P and a new radius t ' 

will be defined. Let ~ I where P^ = i-th data 

point and d^ = distance from P to P^. Also let n(Cp) = the number 

of points in C^. Next, order the N data points by increasing distance 

from P, i.e. d. < d. < ... < d, , and let 
""l " ^2 “ ■ - ^N 

= {P . , Pj , . . . , P . } for k < N. 

^1 2 \ 

Correspondingly, define 

r"(Cp ) = d. 

k+1 



Suppose that each yield estimate should use at least n, 


mxn 


data points but no more than n . Then define 

max 


“P 


C ^min if 0 < n(CT3) < 


n . 
mxn 


if n . < n(C_) < n^^ 

mxn P — max 


C^n^ax if < a(Cj.) 


and Tp = 


>"(0 if 0 < n(C„) < n„. 

p ' — P — mxn 


>Vin ^ “ <=P> i “i 


max 


r^{C ^max) if n < n(C„). 
P max P 


Finally, a new weighting function s (d^) is defined which 
is continuously differentiable for d^>0 and such thats(d^) = 0 
for d^>r''. This is given by 


s . = s (d. ) = 
X X 


1/d. 


if 0 < dj^ j< 2 


d, 


27 if I < di < r" 

1^' (p 3 


) if x'< d^. 

The new yield approximation equation is given by 


f (P) = [Z s.^z.] / S s.^ 

^ J- -L _ -I- 


l.S 




PfECp 


where f2 ~ ® *"P* 


Step 2; Correcting for Direction 
Consider Figure 9 (a) , (b) 
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P. 


41- 

P 


* 


P 


3 


Figure 9 (a) 


— jt ^ If X Figure 9(b) 

Px ^ 


Intuitively, it would seem the yield value at P in Figure 9(a) 
should be closer to the yield at than in Figure 9(b) due to the 
intervening effect of P£ in (b) . A weighted average based strictly 
on distances would provide identical yield estimates at P for 
both configurations. Thus a new directional weighting term is 
derived which represerits this shadowing effect and is denoted by 


t. == [S s. jl - cos (P.PP.:) 
"" P.eCp ^ ^ 


/ 2 
P.ec;: 
3 P 



The distance weighting factor Sj is included in the definition of 
t. since data points near P should have a larger shadowing effect 
than distant data points. Note that 0 _< t^ < 2 for i == 1,2, — ,N 
The final weighted average incorporates this directional 
correlation and is given by (2.8.2). 

Zn o = wrs.] / 2 w: (2.8,2) 

^ P.eC" ^ ^ P.eC" ^ 


where (1) w^ 


s 


2 

i 


X (1 + tx) 
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and ( 2 ) f^(P) = if = 0 for some Pj^ e Cp, 


The resulting yield surface using Method 9 will be continuously 

differentiable and will pass through the N data values. .For 

extremely sparse data situations, we might expect little improvement 

over the average linkage estimators. 

2.9 Method 10 - Modified Least Squares, 

This method, proposed by McLain [3 ], is an attempt to 

improve the linear regression estimator {Method 3) by (1) 

performing a weighted linear regression where the weights are 

functions of the' distances d^ ( 2 ) deriving a different set of 

regression coefficients for each grid point P and (3) introducing 
2 2 

X tY rxy terms into the regression model. For extremely sparse 
data sets one might expect to gain very little by adding the 
quadratic directional effects into the model (as later experiments 
demonstrated). However, the procedure of requiring those data 
points close to P to carry more weight than distant points seemed 
very desirable. Consequently, for each point P, this method 
determines a polynomial of the form 




'00 


+ ""lo^p + ^01^1 


+ c 


20^P 


+ c 


l^XpYp + Co2Yp 


The coefficients are chosen to minimize 
^ 2 

Q = S [f(x- y.) - z.J • w(d.) (2.9.1) 

i=l ^ 1 1 

for some weighting function w(d^) , e.g. w(d^) = 1/d^.. The 
solution to ( 2 . 9 . can be readily obtained by solving the six 
linear equations 

■ 9Q =0 
9C 

rs 



for the six coefficients c 


00" ‘^Ol' °li' °20" ^ 02 ’ 

Having determined these coefficients, then the Method 10 

estimator is given by 

^ P ,10 " '^00 "^10 *^01 ^20 ^11 ^ P^P ^02 

The weighting function w(dj.) is again subjective. McLain 

discusses such functions ranging from w(d ) = 1/d- (slow die— out) 

1 1 

2 2 

to w(dji^) = exp (-ad^ ) / (d^ + g) for suitable constants a and £ 

(fast die-out) . McLain recommends the latter weighting function 
although it should be pointed out that he is assuming a grid 
containing between a hundred and a thousand data points . The 
authors in working with sparse data sets (see Section 3.0) had more 
success in using less drastic weighting functions such as w(dj^) = 
1/dj^ or 1/d^^. 

This method will produce a surface which should better reflect 
yield differences across the enti re r egion R and will pass through 
the N sample data values. By using a different regression equation 
for each grid point, the problem of ill behavior near the boundary 
of R (encountered' in Method 3) is avoided. Overall, we would expect 
a better mean square error using Method 10 over Method 3. 

3.0 An Empirical Study 
3.1 Data Description 

The authors used five years of yield data (bushels/acre) for 
wheat in North Dakota to test the ten extrapolation procedures* . 

The intent of the test was to see how well these methods extrapolated 
across the entire state from data acquired at seven yield stations 
located at Jamestown, Minot, Grafton, Bismark, Dickinson, Fargo, and 

*obtained through NASA contract NAS9-13512 
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Williston. The data existed for the years 1962 - 1966 at these seven 
stations along with yield data at approximately forty five additional 
check points (cities) (Figure 3.1). The methods could then be 
checked for accuracy by applying each to the seven yield observations 
and comparing to the actual yield data at the forty five check points. 
The yield data for these seven stations is contained in Table 3.1. 



1962 

1963 

1964 

1965 

1966 

Bismark 

31.3 

15.9 

20.5 

22.4 

19.1 

Oickinson 

26.1 

22.1 

20.4 

22.2 

23.1 

Fargo 

24.1 

24.0 

— 

24.2 

28.8 

24.7 

Srafton 

28.8 

29.3 

29.8 

30.7 

26.0 

Jamestown 

27.6 

17.1 

mmm 

22.5 

22.7 

\dinot 

34.8 

26.7 

27.4 

28.2 

29.0 

Williston 

28.5 

_ 26.3 

23.. 1 

22.0 

20.3 


Table 3.1 Yield Data (bu/acre) for 
North Dakota Yield Stations 

3.2 Method 9 Results 

In the description of this method, three parameters were defined, 
namely: 

n^^ = the average number of data points contained in a 
circle of radius r 

n . = minimum number of data points to be used in the 


weighted sum 
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^max ~ number of data points to be used in the 

weighted sum. 

%ax equal to 7 with n^^^^ ranging from 2 to 7. The 

results of these computer runs are contained in Table 3.2. In four 
of the five years we obtained a good improvement over the average 
linkage estimates by using Shepard ' s mpdif ications . The adjustment 
to the weights for direction (shadowing) provided a slight improve- 
ment in three of the five years and was actually the best of the 

two of the years. • The value of n-j^ made no difference since 
any value from 2 to 7 provided the same MSE. The ideal value for 
^min varied considerably and there appears to be no ' 

-one optimum" value to use in subsequent applications . 


Year 

Method 6 

Using 

Nearest Points 

Correcting 
for Direction 

1962 

MSE = 11.20 


n^ =-2-7 


n^ = 2-7 



- 

* 

^in =.6.7 


^in “ 6/7 





MSE = 12.10 


MSE = 11.85 


1963 

MSE =7.92 


= 2-7 


■ n^ = 2-7 





^min = 6 ' 


’^min 6/7 

* 




MSE =7.55 


MSE =7.52 


1964 ■ 

MSE = 12.06 


n^ = 2-7 


n^ = 2-7 





^min ~ ^ 


n . =3 

min 

* 




MSE = 10.23 


MSE = 10.16 


1965 

MSE = 7.-61 


n^ = 2-7 


n^^ = 2-7 





n . =4 

mxn 

* 

n^ . = 3 

min 





MSE =5.42 


MSE =5.46 


1966 

MSE =8.17 


n^ = 2-7 


ni - 2-7 





"min ~ 6,7 

* 

n . =5 

min 



*Best for year 


MSE =7.81 


MSE = 7.98 



Table 3.2 Best MSE’s using Method 9 
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3.3 Method 10 Results 

Four models were considered in evaluating the Method 10 
estimator, namely 

Model I: 2 = A 

P 

Model II: §p = A + 'B + C yp 

Model III: 2 „=A+Bx +Cv +Dxv 

P P ' ^P P-^P 

Model IV: gp = A + B Xp + C yp + D Xpyp + E Xp^ + F yp^. 

Five weighting factors were considered in deriving the regression 

coefficients; 1.0, (1/d^) , (1/d^)^, exp (-.002 d^^) , and exp (-ad^^) 
where a = (average distance between data points)”^ and d^ = distance 
from P to data point P^^. For the five sets of data, the Model III 
estimator was found to provide the minimum MSE using a weighting 
factor of 1/d^. This will henceforth be referred to as the Method 
10 estimator. 

3.4 Summary of Results 

Appendix B contains the results of Methods 1-8 and lists the 
actual and approximated results for each of the available check 
points. Table 3.3 summarizes the results of this experiment and 
contains the MSE ' s for each method and for each year . The MSE ' s 
for Method 9 are the smallest of the three MSB's as discussed in 


Section 3.2. 
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Description 

1962 

1963 

1964 

1965 

1966 

1 

Composite Average 

13.84 

16,53 

21.78 

16.55 

12.84 

II 1 

2 

Nearest Neighbor 

14.96 

14.33 

10.69 

7.53 

9.59 

3 

Least Squares 
T.inoar- Siirface 

9.14** 

14.73 

9.00** 

5.95 

7.00** 

4 

LS/NN 

11.46 

9,37 

10.17*** 

6.92 

10.49 

5 

Average Linkage 
(1/d) 

12.00 

10.36 

15.56 

10.98 

9.88 

6 

Average Linkage 
fl/d) ^ 

11.20*** 

7.92*** 

12.06 

7.61 

8.17 

7 

^ -t-/ U. / 

Average Linkage 
with Coordinate 
correl ation 

11.20*** 

9.74 

12.06 

7.61 

8.17 

8 

Meteorological 

Objective 

TPi 1 n rr 

11.65 

7.41* 

10.21 

5.88*** 

8.59 

Q 

MnrHf-jpd Linkaqe 

11,20*** 

7.52** 

10.16 

5.42* 

7.81*** 

10 

Weighted Least 
Squares 

A+Bx+Cy+Dxy 

with wt. = 1/^ 

7.77* 

12.40 

- 8.86* 

5.63** 

6.68* 


Table 3.3 MSB’s for Methods 1-10 


* = smallest ** = 2nd smallest 


*** = 3rd smallest 


3.5 Contour Maps 

The authors wrote a Fortran contour mapping program to further 
analyse the results of the ten extrapolation procedures. Appendix 
C contains 12 such maps. Figures C-1 through C-5 applied Method 10 
to the full set of 52 (or whatever number was available) yreld values 
to obtain the most accurate representation of the Morth Dakota yield 
data for that year. These figures can be compared to Figures C-6 
through C-10 which are the results of applying the best extrapolation 
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procedure (Methods 8,9 or 10) to the set of seven data points as 
discussed in Section 3.1. Figure C-7 is a contour map after 50 
iterations of Step 3 using Method 8 . Figure C-11 demonstrates 
Method 8 applied to the 1963 data immediately after Step 2 (the 
smoothing procedure) . This figure illustrates an early point that 
due to the nature of the data, the basic contours are determined 
after the smoothing process. Finally, Figure C-12 is a contour 
map of Method 3 (one of the more consistently good estimators) 
applied to the 1963 data. 

4.0 Conclusions 

Based on the results of Section 3.4, the superior method for 
extrapolating North Dakota yield data appears to be Method 10 
(Weighted Linear Regression) . It provided the minimum MSE in 
three of the five years. Not far behind are Method 8 (Objective 
Analysis with Low-Pass Filtering Constraints) , Method 3 (Least 
Squares Linear Surface) , and Method -9— (Modified Average Linkage) . 

The Method 3 estimator provided very consistently good MSE's, 

The main point to be made here is that it does appear that the 
more sophisticated methods (8,9,10) appear to work very well on these 
sets of extremely sparse data. It should be pointed out however that 
there was not a lot of variation in the yearly yield data. 

Table 4.1 contains in -Fo c f rr/^r\/^ r\’P 


the ten methods. 
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ethod 




earest 

eighbor 





S/NN 


verage 

Linkage 

kl/d) 


Advantages 

Disadvantages 

. fi 

Provides "reasonable estimates 
over entire region. Yield 
surface is continuous and well 
behaved on boundaries . 

Does not reflect varia- 
tions within the region 
(particularly in small 
neighborhoods about each 
data poin^. 

Provides good estimate in 
small neighborhoods about each 
data point and is well behaved 
on boundaries. 

Surface defined by yield 
estimates is discontinu- 
ous . Maximum yield 
estimate is the maximum 
data, value. Estimate 
completely ignores the 
influence of the other 
N-1 points. Rather poor 
at picking up directional 
trends . 

Provides good estimate in 
convex hull. Extremely good 
at picking up directional 
trends . Yield surface is 
continuous . 

May provide unusually 
large or small (possibly 
negative) estimates on 
boundary of R if slope 
of plane is large. 

Provides good estimate in 
convex hull and reflects 
directional trends in this 
region. Well behaved on 
boundaries. 

Yield surface is discon- 
tinuous . Ignores 
influence of other N-1 
data points outside 
convex hull . 

More accurately models the 
influence of all N data points, 
yet in a small neighborhood 
about data point the yield 

estimate is completely dominated 
by z, . Yield surface is contin- 
uous^and well behaved. 

Maximum yield estimate 
occurs at that data 
point having the largest 
yield. Does not reflect 
directional trends in' 
the data. Yield surface 
has zero gradients at 
data points. 
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Method 

Advantages 

Disadvantages 

6 

Same as Method 5 - in addition 

Same as Method 5 

Average 
Linkage 
(1/d) 2 

the yield estimates appear to 
more accurately reflect the 
relative distances from each 
data point. 

7 

Same as Method 6 - in addition 
reflects significant directional 
correlation 

Could be ill behaved 
near the boundary if ' 
the regression estimators 
have large slope. 

Linkage 

with 

Directional 

Correlation 

8 

Provides a "smooth" well behaved 
surface over R. Considers the 
effect of all M data points. 

Does not exactly fit 
data points . Severely 
affected by smoothing 
procedure , 

Objective 

filtering 

9 

Continuously differentiable . 
Passes through data points . 
Reflects relative distances 
and directions. 

Same as Method 5 

Modified 

Linkage 

10 

Passes through data points. 
Adjusts model for each point 
to be estimated. 

Computationally time 
consuming. 

ifeighted 

Regression 


Table 4 . 1 













179 


Appendix A: A Procedure for Determining the Convex Hull of Two- 

Dimensional Data 



Convex 
Hull (C) 


The convex hull (C) of a set of —two-dimensional data is the 
smallest (in area) convex region containing all of the data points. 

See Figure A. 1 , Given a set of data points Pj,...,Pj^ with coordinates 

' this procedure defines P^^ to be 
(a) an interior point of C if P^ lies within any triangle 

formed by three other data points (P2'^4'^7 Figure A.l) 
or (b) a boundary point of C if P^ lies in no such triangle 

Figure A.l). 

The procedure will be to determine the set of boundary points 
of the input data. Once these points have been determined, it is a • 
relatively easy task to connect this set of points to form the 


convex hull. 
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Thus, it remains to determine if Pj_ lies in any triangle formed 
by three other data points. 




S 


N 




s 




1 ^ ^ 3 

* S 

■ 

\ 


Figure A. 2 


Referring to Figure A. 2, will lie within the triangle formed 
by P 2 /P 3 / and if 


{!) 

^1 

and 

^4 

lie 

on 

the 

same 

side 

of 

^ 1 ^ 

( 2 } 


and 

^3 


on 

the 

same 

side 

of 

^2 

(3) 


and 

^2 

lie 

on 

the 

same 

side 

of 



Care must be taken to. insure that P 2 /P 3 f aud are not colinear. 
If P^ passes the above three tests for all sets of points Pj_,Pj, and 
Pj, (i,j,k f 1), then P^ will be classified as an interior point. The 
Fortran programs required to accomplish this are relatively straight 
forward and consist of 

( 1 ) a routine to determine a matrix containing the possible 

set of subscripts for triangles containing data point P^. 
For example, for N = 5 and data point P^ this matrix is 

12 4 

12 5 

14 5 


2 


4 


5 
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(2) a routine to determine if lies within the triangle 

formed by P. ,P, , and P, . 

D k 1 

The output from these computer programs will be the set of 

boundary points which define the convex hull of the original data 
set. 

When using the Method 4 yield estimation procedure, the above 
technique can be used to determine if a particular grid point P lies 
within the convex hull of the N sample points (in which case one 
would use the linear regression estimator) . The point P will 
lie in the convex hull if it lies within (or on) any triangle 
formed by any three data points and this can be determined using 
the previously defined computer programs. For N = 5, the matrix 
of possible coefficients becomes 

12 3 

12 4 

12 5 

13 4 

13 5 

14 5 

2 3 4 

2 3 5 

2 4 5 

3 4 5 
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Appendix B 


Methods 1-8 Results 
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NORTH DAKOTA SIMULATION - 1962 


FIC.URF, B-1 


estimations 


NO, 

name 

ACTUAL 

ME.THl 

DIFF 

METH2 

DIFF 

METH3 

DIFF 

M£Th4 

DIFF 

1 

AMID 

20.20 

28.74 

-8.5 

26,10 

-5*9 

27.80 

-7,6 

26.10 

-5.9 

2 

ASHL 

23.40 

28.74 

-5.3 

27.60 

-4*2 

25.28 

-1.9 

27,60 

-4.2 

3 

beac 

27.30 

28.74 

-1.4 

26.10 

1.2 

28,89 

-1.6 

26.10 

1.2 

4 

BELC 

30.10 

28.74 

1.4 

34,80 

-4.7 

31.53 

-1.4 

34.80 

-4 • 7 

5 

8EUL 

32.10 

28.74 

3.4 

26.10 

6,0 

28.97 

3,1 

28.97 

3,1 

6 

8IS8 

29.00 

28.74 

0.3 

34,80 

—5 * 8 

30,95 

-1.9 

34.80 

-5,8 

7 

80TT 

32,80 

28.74 

4.1 

34,80 

-2,0 

31.74 

1.1 

34.80 

-2.0 

e 

BOWM 

22.60 

28,74 

-6.1 

26,10 

—3,5 

27.15 

-4.5 

26,10 

-3.5 

9 

CARR 

31.50 

28.74 

2.8 

27.60 

3,9 

28,48 

3.0 

28.48 

3,0 

10 

CARS 

25.00 

28.74 

-3.7 

31.30 

-6,3 

26.94 

-1.9 

31.30 

-6.3 

11 

CAVA 

25.60 

28.74 

-3.1 

28.80 

-3.2 

30,74 

-5,1 

28.80 

-3.2 

12 

CENT 

26.10 

28.74 

-2.6 

31.30 

-5.2 

28,39 

-2,3 

28,39 

-2.3 

13 

COOP 

32.60 

28.74 

3.9 

27.60 

5*0 

27,90 

4.7 

27.90 

4,7 

14 

CROS 

28.20 

28.74 

-0.5 

28.50 

—0 » 3 

32.96 

-4,8 

28,50 

-0.3 

15 

DEVI 

28.00 

28.74 

-0.7 

28.80 

— 0,8 

29.64 

-1.6 

29.. 64 

-1.6 

16 

DUNN 

29,00 

28.74 

0.3 

26.10 

2.9 

29,48 

-0.5 

29.48 

-0.5 

17 

EDGE 

24.30 

28.74 

-4.4 

27.60 

-3,3 

25,94 

-1.6 

27.60 

-3,3 

18 

ELLE 

24.80 

28.74 

-3.9 

27.60 

-2.8 

24.99 

-0.2 

27.60 

-2.8 

19 

FAIR 

27,00 

28.74 

-1.7 

26.10 

0.9 

29.26 

-2.3 

. 26.10 

0,9 

20 

FESS 

31.20 

28.74 

2,5 

27,60 

3,6 

28.84 

2,4 

28.84 

2.4 

21 

FORM 

21.00 

28.74 

-7.7 

24,10 

-3,1 

24,85 

-3,9 

24,10 

-3. 1 

22 

FORT 

3i;oo 

28.74 

2.3 

31.30 

”0*3 

25.93 

5.1 

31.30 

-0.3 

23 

GARR 

31.80 

28.74 

3.1 

34,80 

-3.0 

29.70 

2.1 

29.70 

2.1 

24 

GRAN 

29,90 

28.74 

1.2 

34,80 

-4.9 

30.79 

-0,9 

34,80 

-4.9 

25 

HANK 

17.60 

28.74 

-11.1 

24.10 

-6.5 

24.64 

-7.0 

24, iu 

-6,5 

26 

HILL 

29,00 

28.74 

0.3 

24.10 

4.9 

27,47 

1.5 

27,47 

1.5 

2.7 

KEEN 

29.80 

28.74 

1.1 

28.50 

1.3 

30,71 

-0.9 

30, 7 1 

-0.9 

28 

LANG 

27.80 

28.74 

-0.9 

28,80 

— 1.0 

31.03 

-3.2 

28,80 

-1.0 

29 

LARI 

31.30 

28.74 

2.6 

28.00 

2.5 

28,78 

2.5 

28.78 

2,5 

30 

lint 

27.40 

28,74 

“1,3 

31.30 

-3.9 

26,22 

1.2 

31,30 

-3,9 

31 

LISB 

22.00 

28.74 

-6.7 

24,10 

-2.1 

25.51 

-3,5 

24,1 0 

-2.1 

32 

mado 

30*40 

20.74 

1,7 

34,80 

-4 . 4 

29,71 

0.7 

29.71 

0,7 

33 

MCCL 

28,30 

28.74 

-0.4 

31,30 

— 3 . 0 

28,91 

-0,6 

28.91 

-0,6 

34 

MCHE 

28,50 

28,74 

-0,2 

27.60 

0.9 

28.48 

0.0 

20.<f8 

0,0 

35 

MOHA 

30.90 

28.74 

2,2 

34»8o 

—3 , 9 

32,17 

-1.3 

34.80 

-3,9 

36 

MOTT 

25.00 

28.74 

-3.7 

26,10 

-1*1 

27,23 

-2.2 

28.10 

-1.1 

37 

NAPO 

25.40 

28.74 

-3 . 3 

31,30 

-5.9 

26.51 

-1,1 

31.30 

-5.9 

38 

NEW 

31.00 

28, ?4 

2.3 

31,30 

— 0.3 

27,74 

3,3 

31.30 

-0,3 

39 

PETE 

30.10 

28.74 

1.4 

28.80 

1.3 

29.14 

1.0 

29, 14 

1,0 

40 

RUG8 

29,60 

28.74 

0.9 

34.00 

-5.2 

30.73 

-1 . 1 

34.80 

-5.2 

41 

SHAR 

34,70 

28,74 

6.0 

27.60 

7.1 

28.27 

6.4 

28.27 

6,4 

42 

STAN 

30.40 

28,74 

1,7 

34.80 

-4 » 4 

31,37 

— 1.0 

34.80 

-4,4 

43 

TUTT 

26,70 

28.74 

-2.0 

31,30 

-4.6 

27,89 

-1.2 

27.89 

-1.2 

44 

VALL 

27.90 

28.74 

-0.8 

27.60 

0.3 

26,74 

1.2 

26, 74 

1.2 

average abs, oiff 

0 

2.85 


3,35 


2,42 


2.78 


MSE 


13.84 


14.96 


9,14 


11.46 
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NORTH DAKOTA SIMULATION - 1962 


FiaURF B-'l 


estimations 


NO. 

Name 

actual 

METH5 

DIFF 

METH6 

DIFF 

METH7 

DIFF 

METHa 

OIFF 

1 

AMID 

20.20 

28,25 

-8,1 

27,23 

-7,0 

27.23 

-7.0 

26,20 

-6.0 

2 

ASHL 

23,40 

28,80 

-5,2 

28.57 

-5.2 

28,57 

-5,2 

31.20 

-7.8 

3 

BEAC 

27.30 

28.40 

-1.1 

27,53 

-0,2 

27.53 

-0,2 

26,10 

1.2 

4 

BELC 

30.10 

29,73 

0.4 

30,88 

—0 . 8 

30.88 

-0,8 

33.80 

-3.7 

5 

BEUL 

32.10 

29,24 

2.9 

29.32 

2.8 

29.32 

2.8 

30,10 

2.0 

6 

BISB 

29,00 

29.53 

-0.5 

30.33 

— 1.3 

30.33 

-1.3 

32.00 

-3 • 0 

7 

BOTT 

32.80 

30,12 

2.7 

31.82 

1.0 

31.82 

1.0 

34,50 

-1,7 

8 

80WM 

22.60 

28,49 

-5,9 

27,81 

-5.2 

27,81 

-5,2 

26,10 

-3,5 

9 

CARR 

31,50 

28.85 

2,6 

28.77 

2.7 

28.77 

2.7 

29.60 

1,9 

10 

CARS 

25.00 

29,05 

-4.1 

29.43 

-4,4 

29,43 

-4.4 

30,20 

-5,2 

11 

CAVA 

25,60 

28.84 

-3,2 

28.83 

—3 . 2 

28.83 

-3.2 

28,90 

-3.3 

12 

CENT 

26.10 

29,45 

-3,3 

30.04 

-3.9 

30,04 

-3.9 

30.40 

-4.3 

13 

COOP 

32.60 

28.20 

4,4 

27,67 

4.9 

27.67 

4.9 

27.50 

5,1 

14 

CROS 

28.20 

29,47 

-1.3 

29,81 

-1.6 

29.81 

-1.6 

29,60 

-1.4 

15 

DEVI 

28,00 

29.04 

-1,0 

29,21 

-1*2 

29,21 

-1.2 

30,00 

-2.0 

16 

DUNN 

29,00 

28,82 

0,2 

28.24 

0.8 

28,24 

0.8 

28,70 

0,3 

17 

EDGE 

24.30 

28.28 

-4.0 

27,90 

-3,6 

27.90 

-3.6 

27.80 

-3,5 

18 

ELLE 

24.80 

28.29 

-3.5 

27,91 

-3,1 

27.91 

-3.1 

27,60 

-2.8 

19 

FAIR 

27,00 

28.24 

-1.2 

27,14 

-0.1 

27,14 

-0,1 

26.90 

0,1 

20 

FESS 

31,20 

29.21 

2.0 

29.55 

1.7 

29,55 

1.7 

30.80 

0.6 

21 

FORM 

21.00 

27.85 

-6.8 

26,92 

-5.9 

26,92 

-5.9 

25,10 

-4,1 

22 

FORT 

31,00 

29.01 

2.0 

29.53 

1.5 

29.53 

1.5 

31,30 

-0,3 

23 

6ARR 

31,80 

30.20 

1.6 

31.77 

0.0 

31,77 

0.0 

32.10 

-0,3 

24 

GRAN 

29,90 

31,81 

-1.9 

34,17 

-4.3 

34.17 

-4.3 

34,30 

-4.4 

25 

HANK 

17.60 

27.64 

-10.0 

26.40 

-8.8 

26,40 

-8,8 

24,10 

-6. 5 

26 

HILL 

29,00 

27.44 

1,6 

26,03 

3,0 

26,03 

3.0 

26.00 

3.0 

27 

KEEN 

29,80 

29,17 

0.6 

28,99 

0.8 

28,99 

0,8 

29,50 

0,3 

28 

LANG 

27,80 

29.04 

-1,2 

29,13 

-1.3 

29,13 

-1.3 

29,20 

-1,4 

29 

LARI 

31.30 

28.46 

2.8 

28,34 

3.0 

28,34 

3.0 

28.00 

3.3 

30 

LINT 

27,40 

29.07 

-1,7 

29,65 

—2 . 3 

29,65 

-2.3 

30.90 

-3.5 

31 

LISB 

22,00 

27,58 

-5,6 

26,44 

-4 . 4 

26,44 

-4,4 

25,40 

-3.4 

32 

madd 

30.40 

29,55 

0,8 

30.44 

-0.0 

30,44 

-0.0 

31,80 

-1.2 

33 

MCCL 

28,30 

29.63 

-1.3 

30,46 


30.46 

-2,2 

31,80 

-3,3 

34 

MCHE 

28,50 

28,60 

-0.1 

28,35 

0*2 

28,35 

0,2 

26,80 

-0,1 

35 

moha 

30,90 

30.33 

0.6 

32.16 

-1.3 

32.16 

-1,3 

34.40 

-3,5 

36 

MOTT 

25,00 

28,51 

-3.5 

27,84 

—2 . 8 

27.84 

-2.8 

27,90 

-2,9 

37 

napo 

25,40 

28.93 

-3,5 

29,25 

-3.6 

29,25 

-3.8 

30,00 

-4,6 

38 

NEW 

31.00 

29.54 

1.5 

30,39 

0.6 

30,39 

0.6 

30,20 

0,8 

39 

PETE 

30.10 

28.65 

1.4 

28.60 

1.5 

28,60 

1.5 

28,00 

1.5 

40 

RUGB 

29,60 

30.11 

-0.5 

31.80 

-2.2 

31.80 

-2.2 

33V 1 0 

-3,5 

41 

shar 

34,70 

28,30 

6.4 

27.85 

6.8 

27,85 

6.8 

27,70 

7.0 

42 

STAN 

30,40 

30,02 

0.4 

30,99 

—0 . 6 

30,99 

— 0,6 

31.80 

-1,4 

43 

TUTT 

26.70 

29,23 

-2.5 

29,69 

-3.0 

29,69 

-3,0 

30.30 

-3.6 

44 

VALL 

27.90 

27.71 

0.2 

27,05 

0.9 

27.05 

0.9 

2b, 4o 

1.5 

average ABS, OIFF 

• 

2.64 


2,64 


2.64 


2.84 


MSE 


12.00 


11.20 


11.20 


11.65 
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NORTH DAKOTA SIMULATION - 1963 


estimations 


NO. 

name 

actual 

methi 

DIFF 

METH2 

diff 

METH3 

OIFF 

METH4 

DIFF 

1 

AMID 

20.10 

23,06 

-3.0 

22.10 

-2.0 

16,54 

3.6 

22*10 

-2.0 

2 

ashl 

10.70 

23,06 

-12,4 

17,10 

-6.4 

14,66 

-4,0 

17.10 

-6.4 

3 

BEAC 

20.20 

23.06 

-2,9 

22-, 10 

-1,9 

18, 74 

1.5 

22.10 

-1.9 

4 

BELC 

25,60 

23.06 

2.5 

26, ^0 

-1 . i 

31.10 

-5.5 

26, 7o 

-1.1 

5 

BEUU 

19.10 

23,06 

-4,0 

22.10 

-3.0 

21.70 

-2,6 

21.70 

-2,6 

6 

BISB 

26,50 

23.06 

3,4 

26.70 

-0.2 

30,03 

-3,5 

26, 7o 

-0,2 

7 

0OTT 

24.80 

23,06 

1.7 

26. to 

-1.9 

30.94 

— 6,1 

26, Jo 

-1,9 

8 

BOMB 

25.20 

23,06 

2.1 

26,70 

-1.5 

30,45 

-5.3 

26, fO 

-1,5 

9 

BOWM 

20,20 

23,06 

-2,9 

22.10 

— 1 . 9 

14. 77 

5.^ 

22,10 

-1.9 

10 

CARR 

23.70 

23,06 

0.6 

17,10 

6.6 

23.58 

0,1 

23,58 

0.1 

11 

CARS 

16,50 

23,06 

—6 . 6 

15,90 

0.6 

16,43 

0.1 

15,90 

0.6 

12 

CAVA 

22,00 

23.06 

-1.1 

29,30 

-7.3 

31.69 

-9.7 

29.30 

-7.3 

13 

CENT 

16.00 

23,06 

-7.1 

15,90 

0.1 

20.62 

—4 . 6 

20,02 

-4.6 

14 

COOP 

21,10 

23.06 

-2,0 

17.10 

4.0 

23.26 

-2.2 

23,26 

-2,2 

15 

CROS 

25,40 

23.06 

2,3 

26.30 

-0,9 

30,77 

-5,4 

26,30 

-0,9 

16 

DEVI 

27,20 

23,06 

4,1 

29,30 

-2. 1 

27.23 

-0.0 

27.23 

-0,0 

17 

DUNN 

21,00 

23.06 

-2.1 

22,10 

-1.1 

22,07 

-1,1 

22.07 

-1.1 

18 

EDGE 

16,70 

23,06 

-6.4 

17,10 

— 0.4 

17.19 

-0,5 

17. io 

-0.4 

19 

FAIR 

20,50 

23,06 

-2.6 

22.10 

-1.6 

20,73 

-0,2 

22.10 

-1.6 

20 

FESS 

22.50 

23,06 

-0.6 

17,10 

5.4 

24.06 

-1,6 

24,06 

-1.6 

21 

FORM 

19,00 

23,06 

-4.1 

24.00 

-5.0 

15.74 

3.3 

24,00 

-5.0 

22 

FORT 

15,90 

23.06 

-7,2 

15.90 

0*0 

14,93 

1.0 

15,90 

0.0 

23 

GARR 

23,80 

23,06 

0,7 

26,70 

-2.9 

24,17 

-0.4 

24,17 

-0.4 

24 

GRAN 

19,50 

23,06 

-3.6 

26.70 

—7 . 2 

27.88 

-8,4 

26.70 

-7.2 

25 

HANK 

20.40 

23.06 

-2,7 

24,00 

-3.6 

15,90 

4.5 

24,00 

-3.6 

26 

HILL 

26,40 

23,06 

3,3 

24,00 

2.4 

23,58 

2.8 

23.58 

2.8 

27 

KEEN 

21,50 

23.06 

-1.6 

26,30 

-4.8 

24.92 

-3,4 

24,92 

-3.4 

28 

LANG 

26.50 

23.06 

3,4 

29.30 

-2.8 

31.48 

-5.0 

29, 3o 

r2.e 

29 

LARI 

26.10 

23.06 

3.0 

29,30 

—3 • 2 

26,38 

-0,3 

26,38 

-0.3 

30 

LISB 

17.90 

23,06 

-5,2 

24,00 

—6 « 1 

17,51 

0.4 

24,00 

-6.1 

31 

MADD 

24,00 

23.06 

0,9 

26,70 

-2.7 

26,43 

-2.4 

26,43 

-2.4 

32 

MCCL 

20.10 

23.06 

-3,0 

15.90 

4.2 

23,26 

-3.2 

23.26 

-3.2 

33 

MCHE 

18,70 

23,06 

-4.4 

17,10 

1.5 

24,33 

-5,6 

24,33 

-5,6 

34 

MOHA 

26,20 

23.06 

3,1 

26,70 

-0.5 

30.61 

-4.4 

26,7 0 

-0.5 

35 

MOTT 

20,00 

23.06 

-3,1 

22.10 

-2.1 

16.22 

3,8 

22.10 

-2, 1 

36 

NEW 

18,10 

23,06 

-5.0 

15,90 

2.2 

18.85 

-0.8 

15,90 

2,2 

37 

PETE 

27.60 

23.06 

4.5 

29,30 

-1.7 

26,86 

0.7 

26.86 

0.7 

38 

RUG8 

23.10 

23,06 

0,0 

26,70 

-3.6 

28,68 

-5 , 6 

26.70 

-3.6 

39 

SHAR 

28,20 

23.06 

5.1 

17.10 

11.1 

24.50 

3,7 

24.50 

3.7 

40 

STAN 

26,00 

23,06 

2,9 

26,70 

-0.7 

27,44 

-1,4 

26,70 

-0.7 

41 

VALL 

20,10 

23.06 

-3,0 

17.10 

3.0 

20.36 

-0.3 

20,36 

-0.3 


AVERAGE ABS. OIFF. 3.41 2.9b 3.03 2.35 

MSe 16.53 


14.33 


14.73 


9,57 
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estimations 


NO. 

name 

actual 

METH5 

DIFF 

METH6 

DIFF 

METH7 

DIFF 

METH8 

DIFF 

1 

AMID 

20,10 

22.24 

-2.1 

21,94 

-1.8 

18.92 

1.2 

22,00 

-1,9 

Z 

ASHL 

10,70 

21.22 

-10 .5 

19.55 

-8 . 8 

16.18 

-5.5 

16.00 

-5.3 

3 

BEAC 

20.20 

22.66 

-2.5 

22,47 

-2.3 

20,67 

-0.5 

22. io 

-1,9 

4 

BELC 

25.60 

24,01 

1.6 

25,05 

0.5 

29.00 

-3.4 

27.00 

-1.4 

5 

BEUL 

19,10 

22.10 

-3.0 

21.39 

—2 • 3 

21.90 

-2.8 

21, 50 

-2 . 4 

6 

8ISB 

26,50 

24,01 

2.5 

25.11 

] 

28.23 

-1.7 

27.70 

-1,2 

7 

80TT 

24,60 

24.06 

0,7 

25.19 

-0.** 

29.04 

-4.2 

26,80 

-? • 0 

8 

BOWB 

25,20 

24.10 

1.1 

25,20 

-0.0 

29,04 

-3,8 

26,60 

-1.4 

9 

BOWM 

20,20 

22,18 

-2,0 

21.72 

-1*5 

17,67 

2.5 

22.10 

-1.9 

10 

CARR 

23,70 

21.92 

1,8 

20.59 

3.1 

22.43 

1.3 

21.^0 

2.3 

a 

CARS 

16.50 

20.94 

-4.4 

19,13 

-2.6 

17.63 

-1,1 

17,20 

-0.7 

12 

CAVA 

22.00 

25.56 

-3,6 

28.13 

-6 * 1 

29.98 

-8 • 0 

29,30 

-7.3 

13 

CENT 

16.00 

21.00 

-5,0 

18.99 

-3.0 

20,34 

-4,3 

19,40 

-3,4 

14 

COOP 

21.10 

22.13 

-1,0 

21.02 

O.i 

22.18 

-i.i 

22.20 

-1.1 

15 

CROS 

25.40 

24,05 

1.4 

25.11 

0.3 

29,41 

-4 « 0 

26,40 

-1.0 

16 

DEVI 

27.20 

23.74 

3.5 

24.76 

2.^ 

26.14 

1*1 

26.90 

0.3 

17 

DUNN 

21.00 

22.77 

-1.8 

22.68 

-1.7 

22.71 

-1.7 

23.30 

-2,3 

18 

EDGE 

16.70 

20,83 

-4,1 

16.88 

-2,2 

17.54 

— 0.8 

17.40 

-0.7 

19 • 

fair 

20,50 

22.65 

-2.2 

22.41 

-1.9 

21.84 

-1.3 

22. /O 

-2.2 

20 

FESS 

22,50 

22.23 

0.3 

21.38 

l.i 

23.08 

-0,<} 

22.10 

0,4 

21 

FORM 

19.00 

21,99 

-3,0 

21.34 

-2,3 

17,15 

I. 8 

22.10 

-3,1 

22 

FORT 

15,90 

20.91 

-5,0 

18,88 

-3.0 

16.36 

-0.5 

15.90 

0,0 

23 

garr 

23.80 

23.20 

0.6 

23,95 

-0.1 

24,30 

-0,5 

23.90 

-0.1 

24 

gran 

19,50 

24,79 

-5.3 

26.28 

-6.8 

27,42 

-7.9 

26.50 

-7,0 

25 

HANK 

20.40 

22.33 

-1.9 

22,14 

— 1*7 

17.41 

3,0 

24.00 

-3,6 

26 

HILL 

26,40 

23,21 

3.2 

23,54 

2.9 

22.99 

3,4 

24, 6q 

1.8 

27 

KEEN 

21.50 

24,16 

-2.7 

25,40 

-3.9 

25.56 

-4 * 1 

25, h 

-4.2 

28 

LANG 

26,50 

24.72 

1.8 

26.83 

-0.3 

29,57 

-3,1 

29, io 

-2,6 

29 

LARI 

26,10 

24,34 

1.8 

26,24 

-0.1 

25.83 

0.3 

27,00 

-0,9 

30 

LISB 

17,90 

21.99 

-4.1 

21,51 

-3*6 

18,39 

-0,5 

21,90 

-3,5 

31 

madd 

24.00 

23.24 

0.8 

23.62 

0.4 

25.38 

-1,4 

25.50 

-1.5 

32 

MCCL 

20.10 

22.06 

-2,0 

21,21 

— 1*1 

22,63 

-2,5 

21,40 

-1,3 

33 

MCHE 

18.70 

22.48 

-3.8 

21,75 

-3.1 

23,20 

-4.5 

23,20 

-4,5 

34 

MOHA 

26.20 

24.17 

2.0 

25.39 

0.8 

29,10 

-2,9 

26.70 

-0,5 

35 

MOTT 

20,00 

21.73 

-1,7 

21,12 

-1.1 

18,26 

1,7 

20,00 

0,0 

36 

NEW 

18,10 

20,12 

-2.0 

17.63 

0.5 

18.72 

-0.6 

17,80 

0,3 

37 

PETE 

27,60 

24.38 

3,2 

26,35 

1.3 

26,26 

1.3 

27,20 

0.4 

38 

RUGB 

23,10 

23.90 

-0,8 

24,98 

-1.9 

27,40 

-4.3 

26,80 

-3,7 

39 

SHAR 

28,20 

22.95 

5,3 

22.91 

5.3 

23,57 

4,6 

24,30 

3.9 

40 

STAN 

26,00 

24.19 

1.8 

25,36 

0.6 

27,12 

-1.1 

26.30 

-0,3 

41 

vall 

20.10 

21,30 

-1.2 

19.76 

0,3 

19,80 

0,3' 

20.10 

0.0 

'•average abs, DIPF 

9 

2.65 


2,0 7 


2.47 


2.06 


MSE 


10.36 


7.92 


9,74 


7,41 
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- eSTI 

nations 




NO. 

Name 

ACTUAL 

metki 

DIFF 

METH2 

DIFF 

METH3 

DIFF 

METH4 

DIFF 

1 

AMID 

19,90 

24.16 

-4.3 

20.40 

-0*5 

17,90 

2,0 

20,40 

-0.5 

2 

BELC 

26,90 

24,16 

2.7 

27.40 

— 0,5 

29.75 

-2,8 

27,40 

-0.5 

3 

8EUU 

20,20 

24,16 

-4.0 

20.40 

-0,2 

22.29 

-2.1 

22,29 

-2.1 

4 

BISB 

28,40 

24,16 

4,2 

27,40 

1*0 

29.28 

-0.9 

27.40 

1.0 

5 

BOTT 

28,70 

24,16 

4,5 

27,40 

1*3 

29,28 

-0.6 

27.40 

1.3 

6 

BOWB 

2A,70 

24,16 

0.5 

27,40 

-2,7 

27,88 

-3,2 

27, **0 

-2.7 

7 

BOWM 

17,40 

24.16 

-6.8 

20,40 

-3*0 

16.72 

0.7 

20,40 

-3,0 

8 

CARR 

28,00 

24,16 

3.8 

23,70 

4.3 

25,11 

2.9 

25,11 

2.9 

9 

CARS 

16,00 

24,16 

-8,2 

20.50 

-4,5 

18,91 

-2,9 

20,50 

-4,5 

10 

CAVA 

25,80 

24.16 

1,6 

29,80 

-4*0 

31,47 

-5.7 

29,80 

-4.0 

11 

CENT 

19,30 

24,16 

-4,9 

20.50 

-1,2 

21.82 

-2,5 

21,82 

-2,5 

12 

COOP 

27,30 

24,16 

3.1 

23,70 

3,6 

25.49 

1.8 

25,49 

1.8 

13 

CROS 

22,80 

24.16 

-1,4 

23,10 

-0*3 

27.49 

-4.7 

23. io 

-0.3 

14 

DUNN 

17,60 

24,16 

-6,6 

20,40 

-2,8 

22.06 

-4,5 

22.06 

-4,5 

15 

EDGE 

19,50 

24.16 

-4,7 

23,70 

-4*2 

21.09 

-1,6 

23, 7o 

-4,2 

16 

FAIR 

16,20 

24,16 

-8,0 

20.40 

—4 • 2 

20.80 

-4.6 

20.40 

-4.2 

17 

FESS 

28,70 

24,16 

4,5 

23.70 

■ 5.0 

25.19 

3,5 

25,19 

3,5 

18 

FORM 

19,10 

24.16 

-5.1 

24,20 

-5*1 

20.85 

-1,7 

24.20 

-5.1 

19 

GARR 

25,70 

24,16 

1,5 

27.40 

-1*7 

24,18 

1.5 

24.18 

1,5 

20 

GRAN 

23,10 

24.16 

-1,1 

27.40 

—4 * 3 

27.00 

-3.9 

27.40 

-4,3 

21 

HANK 

18,80 

24,16 

-5,4 

24,20 

—5 * 4 

21.32 

-2.5 

24,20 

-5,4 

22 

HILL 

27,80 

24,16 

3,6 

24,20 

3.6 

26,43 

1,4 

26,43 

1.4 

23 

KEEN 

18.30 

24.16 

-5,9 

23.10 

—4 * 8 

23,71 

-5.4 

23.71 

-5.4 

24 

LANG 

28,90 

24,16 

4.7 

29.80 

-0.9 

30,84 

-1,9 

29.60 

-0,9 

25 

LIS8 

19,40 

24,16 

-4,8 

24,20 

-4*8 

22.03 

-2,6 

24,20 

-4,8 

26 

MADD 

28,10 

24.16 

3.9 

27,40- 

0.7 

26,76 

1.3 

26, 76 

1.3 

27 

MCCL 

22,00 

24,16 

-2.2 

20,50 

1*3 

24,17 

-2.2 

24.17 

-2,2 

28 

MCHE 

25,90 

24,16 

1.7 

23,70 

2.2 

25.97 

-0.1 

25,97 

-0.1 

29 

MOHA 

29,30 

24,16 

5,1 

27,40 

1*9 

28,35 

1.0 

27,40 

1.9 

30 

MOTT 

19,10 

24.16 

-5,1 

20,40 

— 1.3 

18.28 

0.8 

20,40 

-1.3 

31 

NAPO 

15,70 

24.16 

-8,5 

20.50 

-4*8 

20,71 

-5,0 

20,80 

-4.8 

32 

NEW 

19,20 

24,16 

-5.0 

20.50 

-1*3 

20.64 

-1,4 

20,50 

-1.3 

33 

RUGB 

24,20 

24,1b 

0,0 

27.40 

-3.2 

28.02 

-3,8 

27,40 

-3.2 

34 

SHAR 

29,80 

24.16 

5,6 

23,70 

6*1 

26,44 

3.4 

26,44 

3.4 

35 

STAN 

25,70 

24,16 

1.5 

27,40 

-1.7 

25.75 

-0,1 

27.40 

-1.7 

36 

TUTT 

16,70 

24,16 

-7,5 

20,50 

-3,8 

22.91 

-6,2 

22, 9i 

-6,2 

37 

VALL 

23,70 

24.16 

-0.5 

23,70 

0.0 

23,68 

0.0 

23,68 

0.0 

average abs. diff 

« 

4.12 


2.77 


2.52 


2.69 

MSE 



21.78 


10.69 


9.00 


10.17 



c - :? 
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estimations 


NO. 

name 

actual 

METH5 

DIFF 

MLTH6 

DIFF 

METH7 

DIFF 

METH8 

DIFF 

1 

AMID 

19,90 

22.54 

-2,6 

21.21 

— 1,3 

21.21 

-1.3 


-0.5 

2 

BELC 

26.90 

25,09 

1.8 

26,06 

0,8 

26,06 

0.8 

27.70 

-0.8 

3 

BEUL 

20.20 

23,13 

"2.9 

22,31 

-2.1 

22.31 

-2.1 

22,40 

-2,2 

4 

BISB 

28,40 

25,20 

3,2 

26,27 

2.1 

26,27 

2,1 

28,40 

0,0 

5 

BOTT 

28.70 

25,02 

3.7 

26.01 

2.7 

26,01 

2.7 

27,50 

1.2 

6 

BOWB 

24.70 

24.51 

0.2 

25.08 

— 0 0 4 

25,08 

-0.4 

26,80 

-2,1 

7 

BOWM 

17,40 

22,82 

-5,4 

21.64 

-4.2 

21,64 

-4,2 

20,40 

-3.0 

8 

CARR 

28,00 

24,25 

3,8 

24,17 

3.8 

24.17 

3,8 

25,30 

2.7 

9 

CARS 

16,00 

22.65 

-6,6 

21.45 

—5 . 5 

21,45 

-5,5 

20,50 

-4,5 

10 

CAVA 

25.80 

26.61 

-0,8 

28.84- 

-3.0 

28,84 

-3.0 

29,80 

-4,0 

11 

CENT 

19,30 

22,78 

-3,5 

21,62 

-2.3 

21.62 

-2,3 

21. <30 

-2,3 

12 

COOP 

27,30 

24.48 

2.8 

24,49 

2.8 

24,49 

2.8 

25,50 

1.8 

13 

CROS 

22.80 

24,10 

-1,3 

24,04 

-1.2 

24„04 

-1.2 

23,80 

-1.0 

14 

DUNN 

17,60 

23,01 

-5.4 

22,07 

-4.5 

22,07 

-4.5 

22.20 

-4,6 

15 

EDGE 

19.50 

23,75 

-4.2 

23,54 

-4.0 

23,54 

-4,0 

23,30 

-3,8 

16 

FAIR 

16.20 

22.52 

-6,3 

21,24 

—5 . 0 

21,24 

-5.0 

21.00 

-4,8 

17 

FESS 

28,70 

24,25 

4,4 

24,25 

4*^ 

24.25 

4,4 

25,40 

3.3 

18 

FORM 

19.10 

24,02 

-4,9 

23.93 

—4 . 8 

23.93 

—4 * 8 

24.10 

-5,0 

19 

GARR 

25,70 

24 ,23 

1,5 

24.88 

0.8 

24,88 

0.8 

24,90 

0.8 

20 

GRAN 

23.10 

25.68 

-2.6 

27,01 

-3.9 

27.01 

-3.9 

27,30 

-4.2 

21 

HANK 

18,80 

24,13 

-5,3 

24.09 

-6.3 

24,09 

-5,3 

24,20 

-5.4 

22 

HILL 

27,80 

24.74 

3.1 

24,84 

3.'0 

24.84 

3,0 

25.70 

2.1 

23 

KEEN 

18,30 

23.58 

-5,3 

23,30 

-5.0 

23.30 

-5.0 

23.60 

-5,3 

24 

LANG 

28,90 

25,87 

3,0 

27.77 

1.1 

27.77 

1.1 

29.60 

-0,7 

25 

LISB 

19,40 

24.08 

-4.7 

24.02 

-4 . 6 

24,02 

— 4,6 

24,00 

—4,6 

26 

MADD 

28,10 

24,77 

3,3 

25.38 

2.7 

25,38 

2,7 

27,20 

0.9 

27 

MCCL 

22.00 

23,89 

-1.9 

23,66 

-1.7 

23,66 

-1,7 

24,30 

-2,3 

28 

MCHE 

25,90 

24.61 

1,3 

24.82 

1.1 

24.82 

1.1 

26,30 

-0,4 

29 

MOHA 

29,30 

24,82 

4.5 

25,77 

3,5 

25,77 

3,5 

27,20 

2,1 

30 

MOTT 

19,10 

22.56 

-3,5 

21,30 

-2,2 

21,30 

-2,2 

20.40 

-1,3 

31 

napo 

15.70 

23.23 

-7,5 

22.48 

—6 .'8 

22.48 

—6, 8 

21.60 

-5.9 

32 

NEW 

19,20 

22.33 

-3.1 

21.05 

-1.9 

21,05 

-1.9 

20.60 

-1.4 

33 

RUGB 

24,20 

25.06 

-0,9 

26.06 

-1.9 

26.06 

-1,9 

27, To 

-3.5 

34 

SHAR 

29,80 

24.86 

4,9 

25,34 

4.5 

25,34 

4,5 

26, 6o 

3.2 

35 

STAN 

25,70 

24,32 

1,4 

24,74 

1.0 

24,74 

1.0 

25,30 

0,4 

36 

TUTT 

16,70 

23.34 

-6 , 6 

22.50 

—5 . 8 

22.50 

-5.8 

22.70 

-6,0 

37 

VALL 

23,70 

24.11 

-0.4 

23.95 

-O.ci 

23,95 

-0,2 

24.00 

-0.3 


average ABS. DIFF. 3.48 3,03 3.03 2.66 

MSE 


15.56 


1H.06 


12.06 


10.21 
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PIGURF. P-7 


estimations 


NO. 

name 

ACTUAL 

methi 

DIFF 

METH2 

DIFF 

METH3 

DIFF 

METH4 

DIFF 

1 

AMID 

21,90 

25,26 

^3,4 

22.20 

-0*3 

18.93 

3.0 

22.20 

-0.3 

2 

ASHL 

18,60 

25.26 

-6,7 

22,50 

-3.9 

21.79 

-3,2 

22.50 

-3.9 

3 

BEAC 

22.30 

25.26 

-3,6 

22,20 

0.1 

19,47 

2.8 

22.20 

0.1 

A 

8ELC 

26,70 

25,26 

1,4 

28,20 

— 1.5 

29.69 

-3.0 

28,20 

-1,5 

5 

BEUU 

24,50 

25,26 

-0.8 

22.20 

2.3 

23,05 

1.4 

23.05 

1.4 

6 

BISB 

28.90 

25,26 

3,6 

28.20 

0.7 

29.53 

-0.6 

28.20 

0.7 

7 

bott 

29,20 

25,26 

3.9 

28,20 

1.0 

29,04 

0,2 

28.20 

1 .0 

8 

BOWB 

30,00 

25.26 

4,7 

28,20 

1.8 

27,09 

2,9 

28,20 

1.8 

9 

BOWM 

18,80 

25.26 

-6 , 5 

22.20 

-3.4 

18.04 

0,8 

22.20 

-3 • 4 

10 

CARR 

23,30 

25,26 

-2,0 

22,50 

0.8 

26.47 

-3,2 

26,47 

-3.2 

11 

CARS 

20.50 

25,26 

-4.8 

22.40 

-1,9 

20.59 

-0.1 

22. ‘♦0 

-1.9 

12 

CAVA 

29,90 

25.26 

4.6 

30.70 

— 0*8 

32.08 

-2,2 

30.70 

-0.8 

13 

CENT 

21.50 

25,26 

-3.8 

22,40 

-0.9 

22.89 

-1.4 

22.89 

-1.4 

14 

COOP 

28,20 

25.26 

2.9 

22.50 

5.7 

27,26 

0.9 

27,26 

0.9 

15 

CROS 

27,20 

25.26 

1.9 

22.00 

5.2 

26,30 

0,9 

22.00 

5.2 

16 

DEVI 

31.30 

25,26 

6.0 

30.70 

0*6 

28,69 

2.6 

28,69 

2,6 

17 

DUNN 

22.50 

25,26 

-2.6 

22.20 

0*3 

22,48 

0.0 

22.48 

0.0 

18 

EDGE 

24,60 

25,26 

-0.7 

22.50 

2.1 

23.63 

1.0 

22.50 

2.1 

19 

ELLE 

20,90 

25,26 

-4 , 4 

22,50 

— 1.6 

22.65 

-1.8 

22.50 

-1.6 

20 

■ FAIR 

22.00 

25,26 

-3,3 

22.20 

—0 . 2 

21.23 

0,8 

22,«:o 

-0.2 

21 

FESS 

24,40 

25.26 

-0.9 

22.50 

1.9 

26.33 

-1.9 

26,33 

-1,9 

22 

FORM 

25,80 

25,26 

0.5 

28.80 

-3.0 

24.04 

1.8 

28,80 

-3.0 

23 

FORT 

20.20 

25.26 

-5.1 

22.40 

-2.2 

20.78 

-0.6 

22.40 

-2.2 

24 

6ARR 

28,70 

25,26 

3.4 

28,20 

0.5 

24,68 

4.0 

24,68 

4.0 

25 

GRAN 

24.30 

25.2b 

-1,0 

28.20 

-3.9 

27.12 

-2,8 

28,20 

-3.9 

26 

hank 

25.70 

25.26 

0.4 

28.80 

-3.1 

24,69 

1.0 

28,80 

-3.1 

27 

hill 

32.20 

25.26 

6.9 

28,80 

3.4 

28.56 

3,6 

28,56 

3.6 

28 

KEEN 

23,10 

25.26 

-2.2 

22.00 

1.1 

23,54 

-0.4 

23,54 

-0.4 

29 

LANG 

33,30 

25.26 

8,0 

30.70 

2.6 

31.21 

2.1 

30, /O 

2.6 

30 

LARI 

30,90 

25,26 

5,6 

30,70 

0*2 

29,40 

1.5 

29,40 

1 .5 

31 

LINT 

22,20 

25,26 

-3,1 

22,40 

-0.2 

21,81 

0,4 

22,40 

-0.2 

32 

LISB 

24,60 

25,26 

-0,7 

28.80 

—4 . 2 

24,93 

-0.3 

28.80 

-4.2 

33 

MADD 

25,70 

25.26 

0,4 

28,20 

— 2 . 5 

27,53 

-1.8 

27,53 

-1.8 

34 

MCCL 

22.50 

25,26 

-2,8 

22.40 

0 « 1 

25,17 

-2.7 

25,17 

-2.7 

35 

MCHE 

22,70 

25.26 

-2,6 

22.50 

0.2 

27,42 

-4,7 

27,42 

-4.7 

36 

MOHA 

30,10 

25.26 

4.8 

28.20 

1.9 

27.74 

2.4 

28,20 

1.9 

37 

MOTT 

23,00 

25.26 

-2,3 

22,20 

0.8 

19,72 

3,3 

22.40 

0.0 

38 

NAPO 

19,60 

25.26 

-5,7 

22,40 

-2.8 

22,84 

-3,2 

22,40 

-2,8 

39 

NEM 

22.10 

25,26 

-3.2 

22.40 

— 0 * 3 

22,00 

O.I 

22.40 

-0.3 

40 

PETE 

34,40 

25.26 

9.1 

30,70 

3.7 

29,26 

5,1 

29.26 

5.1 

41 

RUG8 

23.60 

25,26 

-1.5 

28,20 

—4 » 4 

28,28 

-4,5 

28.20 

-4.4 

42 

SHAR 

31.20 

25,26 

5,9 

22,50 

8.7 

28.07 

3.1 

28,07 

3,1 

43 

STAN 

27,40 

25,26 

2.1 

28,20 

— 0 « 8 

25,38 

2.0 

28,20 

-0.8 

44 

TUTT 

20.20 

25.26 

-5.1 

22,40 

-2.2 

24,41 

—4 , 2 

24.41 

-4,2 

45 

VALL 

26,30 

25.26 

1.0 

22.50 

3.8 

25.99 

0,3 

25,99 

0.3 

average 

ABS, DIFF 


3,45 


2,08 


2,02 


2,17 


MS£ 


16«55 


7.53 


5.95 


6,92 
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FIGUB}-] B-Q 


estimations 


NO, 

name 

ACTUAL 

METH5 

DIFF 

METH6 

DIFF 

METH7 

DIFF 

METH8 

DIFF 

I 

AMID 

21,90 

23.74 

■■1.8 

22,71 

-0*8 

22.71 

-0.8 

22.20 

-0,3 

2 

ASHL 

18.60 

24.63 

-6,0 

23.95 

-5.4 

23,95 

-5,4 

22. ‘fO 

-3.8 

3 

beac 

22,30 

23,82 

-1.5 

22.83 

-0.5 

22,83 

-0.5 

22,80 

0.1 

A 

8ELC 

26,70 

26,04 

0,7 

26,91 

-0.2 

26.91 

-0,2 

28,50 

- r ,8 

5 

BEUL 

24,50 

24.21 

0,3 

23.56 

0.9 

23,56 

0,9 

23,70 

0,8 

6 

BISS 

28,90 

26.19 

2,7 

27,17 

1.7 

27,17 

1.7 

29,20 

-0,3 

7 

BOTT 

29,20 

25,93 

3,3 

26.80 

2.4 

26.80 

2,4 

26.30 

0.9 

8 

BOWS 

30.00 

25.20 

4,8 

25.50 

4.5 

25.50 

4.5 

27.30 

2.7 

9 

BOWM 

18,80 

23,98 

-5.2 

23.01 

-4.2 

23,01 

-4,2 

22.20 

-3 . 4 

10 

CARR 

23.30 

25,13 

-1,8 

24,63 

-1.3 

24.63 

-1.3 

25,40 

-2.1 

11 

CARS 

20.50 

23,90 

-3,4 

22,97 

-2.5 

22.97 

-2,5 

22,80 

- 1.8 

12 

CAVA 

29,90 

27.67 

2.2 

29,81 

0.1 

29,81 

0.1 

30,70 

-0.8 

•13 

CENT 

21,50 

23.99 

-2,5 

23.12 

-1.6 

23.12 

-1,6 

23.20 

-1,7 

14 

COOP 

28,20 

25,55 

2.7 

25,26 

2.9 

25,26 

2,9 

26,30 

1.9 

15 

CROS 

27,20 

24 , 64 

2.6 

24,03 

3.2 

24,03 

3,2 

23, UO 

4.2 

16 

DEVI 

• 31.30 

26,27 

5.0 

27,32 

4.0 

27,32 

4,0 

28,90 

2.4 

17 

DUNN 

22,50 

23,98 

-1 • 5 

23,16 

-0*7 

23,16 

-0,7 

23.10 

-0,6 

18 

EDGE 

24,60 

24,60 

-0.0 

23.72 

0.9 

23,72 

0,9 

22.90 

1.7 

19 

ELLE 

20,90 

24.93 

-4.0 

24,49 

—3 . 6 

24.49 

-3 , 6 

22.80 

-1.7 

20 

FAIR 

22,00 

23.61 

-1.6 

22.63 

—0 . 8 

22.63 

-0,6 

22, ‘fO 

-0.4 

21 

FESS 

24.40 

25,19 

-0.8 

24,95 

—0 . 6 

24.95 

-0,6 

25, 70 

-1,3 

22 

FORM 

25,60 

25.49 

0.3 

25,73 

0.1 

25,73 

0.1 

27,10 

-1,3 

23 

FORT 

20,20 

24.23 

-4,0 

23,34 

-3.1 

23.34 

-3,1 

22.40 

-2.2 

24 

GARR 

28,70 

25,15 

3.6 

25,76 

2.9 

25,76 

2.9 

25.80 

2.9 

25 

GRAN 

24.30 

26,53 

-2.2 

27,81 

-3.5 

27.81 

-3,5 

28,10 

-3,8 

26 

HANK 

25.70 

25,81 

-0.1 

26,49 

-0.6 

26.49 

-0.8 

28,80 

-3. 1 

27 

HILL 

32,20 

26,62 

5.6 

27,67 

4.5 

2 7,87 

4,5 

26,60 

3.6 

28 

KEEN 

23,10 

23.88 

-0.8 

22.85 

0.2 

22.85 

0.2 

23,20 

— 0 * 1 

29 

LANG 

33,30 

26,91 

6.4 

28,74 

4,6 

28,74 

4,6 

30,50 

2,8 

30 

LARI 

30,90 

26.99 

3,9 

28,67 

2.2 

28,67 

2.2 

29.40 

1.5 

31 

LINT 

22,20 

24,19 

-2,0 

23,26 

-1.1 

23,26 

-1,1 

22,‘fO 

-0.2 

32 

LIS8 

24,60 

25.67 

-1.1 

26,08 

-1*5 

26,08 

-1,5 

26, 5o 

-1,9 

33 

MADD 

25,70 

25,74 

-0.0 

26,23 

-0.5 

26,23 

-0,5 

27,80 

-2. 1 

34 

MCCL 

22,50 

24.89 

-2.4 

24,64 

-2.1 

24,64 

-2.1 

25,10 

-2,6 

35 

MCHE 

22,70 

25.61 

-2,9 

25,53 

-2.8 

25,53 

-2,0 

28,70 

-4,0 

36 

MOHA 

30,10 

25,60 

4,5 

26.41 

3 • T 

26,41 

3.7 

27.90 

2,2 

37 

MOTT 

23.00 

23.80 

-0.8 

22,82 

0.2 

22.82 

0.2 

22.20 

0.8 

36 

NAPO 

19,60 

24,22 

-4,6 

23.29 

-3.7 

23,29 

-3.7 

22.40 

-2,8 

39 

NEM 

22,10 

23.67 

-1.6 

22.73 

-0.8 

22,73 

-0,6 

22.50 

-0,4 

40 

PETE 

34,40 

26,93 

7,5 

28.65 

5.7 

28,65 

5.7 

29,40 

5,0 

41 

RUGB 

23,80 

26.0 0 

-2.2 

26,89 

— 3 . 1 

26,89 

-3,1 

28.50 

-4,7 

42 

SHAR 

31.20 

26.06 

5,1 

26,51 

4.7 

26,51 

4,7 

27.70 

3,5 

43 

STAN 

27,40 

24,89 

2.5 

24,95 

2.5 

24,95 

2.5 

25, Jo 

2.1 

44 

TUTT 

20,20 

24,32 

-4,1 

23.41 

— 3 . c 

23.41 

-3.2 

23, 20 

•■3 • 0 

45 

VALL 

26,30 

25,20 

1.1 

24,62 

1.7 

24.62 

1.7 

25,20 

1,1 

AV6HA6E A8S. DIFF 

• 

2,75 


2,2b 


2.26 


2,05 

MSE 



10.98 


7,61 


7,61 


5,88 
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FTauirn 








— ESTI 

nations 




NO* 

name 

ACTUAL 

methi 

DIFF 

METH2 

DIFF 

METH3 

DIFF 

METH4 

DIFF 

1 

amid 

25.60 

23.56 

2.0 

23.10 

2,5 

19*52 

6.1 

23.10 

2.5 

2 

ASHL 

16*40 

23,56 

-7,2 

22.70 

•”6,3 

20*42 

-4,0 

22, 7o 

-6,3 

3 

BEAC 

22*80 

23,56 

-0.8 

23,10 

-0,3 

20.16 

2,6 

23,10 

-0.3 

A 

BELC 

25.00 

23*56 

1,4 

29,00 

-4,U 

27,05 

-2.1 

29,00 

-4.0 

5 

BEUL 

22*50 

23.56 

-1,1 

23.10 

— 0 , 5 

22.32 

0.2 

22,32 

0,2 

6 

BISB 

26*90 

23.56 

3.3 

29,00 

—2 * 1 

26 . 7.8 

0,1 

29.00 

-2.1 

7 

BOTT 

28*00 

23,56 

4,4 

29.00 

-1.0 

26,74 

1,3 

29,00 

-1 . 0 

8 

80WB 

25,70 

23.56 

2.1 

29,00 

-3,3 

25,78 

-0,1 

29,00 

—3 . 3 

9 

BOWM 

16,40 

23.56 

-7.2 

23.10 

-6,7 

18.78 

-2.4 

23. io 

-6.7 

10 

CARR 

23,60 

23,56 

0.0 

22.70 

0,9 

24,19 

-0,6 

24,19 

-0.6 

n 

CARS 

21.50 

23.56 

-2,1 

19,10 

2,4 

20,23 

1.3 

19.10 

2,4 

12 

CAVA 

24,30 

23,56 

0.7 

26,00 

-1.7 

28,22 

-3,9 

26,00 

-1.7 

13 

CENT 

17*10 

23,56 

-6.5 

19.10 

-2,0 

22,05 

-4,9 

22.05 

-4,9 

lA 

COOP 

24,10 

23,56 

0.5 

22.70 

1*4 

24,48 

-0.4 

24,48 


15 

CROS 

24,60 

23.56 

1.0 

20,30 

4,3 

25,50 

-0,9 

20,30 

4.3 

16 

DEVI 

27*20 

23,56 

3,6 

26,00 

l.ii 

25,87 

1.3 

25,87 

1.3 

17 

DUNN 

20*50 

23,56 

-3,1 

23,10 

-2,8 

22,14 

-1,6 

22,14 

-1.6 

18 

EDGE 

21*20 

23,56 

-2,4 

22.70 

-1.5 

21,71 

-0*5 

22. 7 0 

-1.5 

19 

ELLE 

18*10 

23,56 

-5,5 

22. 70 

—4 • 8 

20,84 

-2.7 

22.70 

-4.6 

20 

FAIR 

20*50 

23*56 

-3.1 

23.10 

-2.6 

21,34 

-0.8 

23.10 

-2.6 

21 

FESS 

21.30 

23,56 

-2,3 

22,70 

-1,4 

24.22 

-2.9 

24,22 

-2,9 

22 

FORM 

21,90 

23,56 

-1,7 

24,70 

-2,8 

21,62 

0.3 

24,70 

-2.8 

23 

FORT 

17*50 

23,56 

-6,1 

19.10 

-1,8 

20.02 

-2*5 

19.10 

-1.6 

2A 

GARR 

24,00 

23,56 

0,4 

29,00 

-s.o 

23.52 

0.5 

23,52 

0,5 

25 

GRAN 

22.20 

23*56 

-1,4 

29,00 

—6 , 8 

25,30 

-3*1 

29,00 

-6.8 

26 

HANK 

21*10 

23*56 

-2,5 

24.70 

-3*8 

21.93 

-0.8 

24,70 

-3.6 

27 

HILL 

25,00 

23,56 

1.4 

24.70 

0,3 

25,11 

-0,1 

25,11 

-0,1 

28 

KEEN 

21*10 

23,56 

-2,5 

20,30 

0,8 

23,16 

-2.1 

23. l6 

-2,1 

29 

LANG 

28.40 

23,56 

4.8 

26,00 

2*4 

27,80 

0,6 

26,00 

2.4 

30 

LARI 

24.10 

23*56 

0.5 

26.00 

-1.9 

26.02 

-1,9 

28,02 

-1.9 

31 

LINT 

ia*5o 

23.56 

-5,1 

19.10 

-0,8 

20.73 

-2,2 

19, io 

-0.6 

32 

LISB 

21*30 

23.56 

-2.3 

24,70 

-3,4 

22.35 

-1.0 

24,70 

-3.4 

33 

MADO 

24.50 

23,56 

0,9 

29.00 

-4,5 

25,20 

-0,7 

25,20 

-0,7 

3A 

MCCL 

17.50 

23,56 

-6,1 

19.10 

-1.6 

23,56 

-6,1 

23,56 

-6.1 

36 

MCHE 

21*60 

23,56 

-1,8 

22*70 

-0,9 

24,75 

-3.0 

24.75 

-3,0 

36 

MOHA 

29,50 

23,56 

5,9 

29,00 

0.5 

26,10 

3,4 

29.00 

0,5 

37 

MOTT 

24,00 

23,56 

0,4 

23,10 

0,9 

19,80 

4,2 

23,10 

0,9 

38 

NAPO 

18,60 

23,56 

-5.0 

19,10 

-0*5 

21.43 

-2.8 

19.10 

-0,5 

39 

NEW 

18,50 

23,56 

-5.1 

19,10 

-0,8 

21.31 

—2 , 8 

19.10 

-0.6 

AO 

PETE 

28,40 

23,56 

4,8 

26,00 

2,4 

26,05 

2,3 

26 ,05 

2.3 

41 

RUGB 

22.00 

23,56 

-1*6 

29,00 

—7 » 0 

25,97 

-4,0 

29,00 

-7.0 

42 

SHAR 

25.60 

23,56 

2.0 

22.70 

2,9 

25,07 

0.6 

25,07 

0,5 

43 

STAN 

24,50 

23.56 

0,9 

29,00 

-4.5 

24,45 

0.0 

29,00 

-4.5 

AA 

TUTT 

17.00 

23,56 

-6.6 

19,10 

-2,1 

22.79 

-5.8 

22,79 

-5.8 

A5 

VALL 

24*70 

23.56 

1.1 

22,70 

2*0 

23.36 

1,3 

23,36 

1.3 

average ABS. DIFF 

• 

2.91 


2.51 


2.07 


2,55 

MSE 



12,84 


9,59 


7,00 


10,49 
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NORTH DAKOTA SIMULATION - 1966 


Finiij^r 5-10 


estimations 


no* 

name 

ACTUAL 

METH5 

OIFF 

METH6 

OIFF 

METH7 

DIFF 

METH8 

DIFF 

1 

amid 

25.60 

22.96 

2.6 

22.81 

2.8 

22.81 

2,8 

23.00 

2.6 

2 

ASHL 

16,40 

22,98 

”6 . 6 

22.40 

— 6 . 0 

22,40 

-6.0 

19, JO 

-2,9 

3 

BEAC 

22.80 

22.96 

-0,2 

22.75 

O.i 

22.75 

0.1 

23, iO 

-0.3 

4 

8EUC 

25,00 

24,49 

0,5 

25,63 

— 0.6 

25,63 

-0,6 

28,50 

-3.5 

5 

BEUL 

22.50 

23.07 

-0,6 

22.70 

-0*2 

22.70 

-0.2 

23,00 

-0.5 

6 

BISB 

26,90 

24,45 

2,5 

25.41 

l.i> 

25,41 

1,5 

27,60 

-0.7 

7 

BOTT 

28*00 

24,66 

3.3 

26,19 

1,6 

26,19 

1.8 

28.90 

-0.9 

8 

BOMB 

25,70 

24.09 

1.6 

24.92 

0.8 

24,92 

0.8 

27,70 

-2.0 

9 

bOWM 

16,40 

22.95 

-6.6 

22.66 

—6.3 

22.66 

-6.3 

23.10 

-6.7 

10 

CARR 

23.60 

23.56 

0.0 

23,37 

0.2 

23,37 

0,2 

24,20 

-0,6 

n 

CARS 

21.50 

22.40 

-0,9 

21,29 

0.2 

21.29 

0,2 

19,90 

1,6 

12 

CAVA 

24.30 

24,76 

-0,5 

25,68 

-1,4 

25,68 

-1,4 

26,00 

-1,7 

13 

CENT 

17,10 

22.47 

-5,4 

21.23 

-4 . 1 

21.23 

-4.1 

21.70 

-4.6 

14 

COOP 

24.10 

23,70 

0.4 

23.61 

0.5 

23,61 

0.5 

24.10 

0.0 

15 

CROS 

24,60 

23,37 

1.2 

22.95 

1.7 

22.95 

1,7 

21,80 

2,8 

16 

DEVI 

27,20 

24.24 

3.0 

24,86 

2.3 

24,86 

2.3 

26,20 

1,0 

17 

DUNN 

20.50 

23,10 

-2,6 

22,89 

-2,4 

22.89 

-2.4 

23,00 

-2.5 

18 

EDGE 

21.20 

23.11 

-1.9 

22,76 

“1,6 

22.76 

-1.6 

22,40 

-1.2 

19 

ELLE 

18.10 

23.22 

-5.1 

22.92 

—4 » 8 

22.92 

-4.8 

22, 7o 

-4,6 

20 

fair 

20,50 

22.98 

“2,5 

22.86 

-2.4 

22.86 

-2.4 

22.90 

-2.4 

21 

FESS 

21.30 

23.66 

-2.4 

23,66 

-2*4 

23.66 

-2.4 

24,80 

-3,5 

22 

FORM 

21.90 

23,52 

-1,6 

23,55 

-1.6 

23,55 

-1,6 

24,10 

-2,2 

23 

FORT 

17,50 

22.56 

-5,1 

21,42 

-3.9 

21.42 

-3 , 9 

19.10 

-1,6 

24 

6ARR 

24,00 

24,16 

-0.2 

25,39 

-1.4 

25.39 

-1.4 

25.60 

-1,6 

25 

GRAN 

22,20 

26,08 

-3,9 

28.34 

-6*1 

28.34 

-6.1 

28,70 

-6,5 

26 

hank 

21.10 

23.66 

-2,6 

23,86 

-2,8 

23,86 

-2,8 

24, 7 0 

-3, fa 

27 

hill 

25,00 

24.08 

0.9 

24,42 

0.6 

24,42 

0,6 

24,80 

0.2 

28 

KEEN 

21,10 

22.68 

-1,6 

21,57 

-0.5 

21.57 

— 0,5 

22.30 

-1.2 

29 

LANG 

28.40 

24.52 

3.9 

25.39 

3.0 

25.39 

3.0 

26,20 

2.2 

30 

■LARI 

24,10 

24.39 

-0 . 3 

25.12 

-1.0 

25.12 

-1.0 

25.40 

-1.3 

31 

LINT 

18,50 

22.47 

-4.0 

21,24 

— 2.7 

21.24 

-2. 7 

19,50 

-1,0 

32 

LISB 

21.30 

23.60 

-2,3 

23.73 

-2.4 

23,73 

-2.4 

23,90 

-2.6 

33 

MADD 

24,50 

24.21 

0.3 

24.97 

-0.5 

24.97 

-0.5 

26,70 

-2.2 

34 

MCCL 

17,50 

23.48 

-6,0 

23.36 

-5*9 

23.36 

-5.9 

24,40 

-6.9 

35 

MCHE 

21.80 

23,79 

-2.0 

23.80 

-2.0 

23,80 

—2 , 0 

24.60 

-2.8 

36 

MOHA 

29,50 

24.57 

4,9 

26.13 

3.4 

26, 13 

3.4 

28,60 

0.9 

37 

MOTT 

24.00 

22.78 

1.2 

22.43 

1.6 

22.43 

1.6 

21,70 

2.3 

38 

NAPO 

18.60 

22.58 

-4,0 

21.61 

-3.0 

21,61 

-3,0 

20,40 

-1.8 

39 

NEW 

18,50 

21,89 

-3.4 

20.29 

-1,8 

20.29 

-1,8 

20,40 

-1.9 

40 

PETE 

28.40 

24.40 

4.0 

25.17 

3*2 

25.17 

3,2 

25,60 

2,8 

41 

RUGS 

22,00 

24,69 

-2.7 

26,22 

-4,2 

26,22 

-4.2 

28,20 

-6,2 

42 

SHAR 

25.60 

23.95 

1.7 

24,15 

1,4 

24,15 

1,4 

24. 70 

0,9 

43 

STAN 

24,50 

23.88 

0.6 

24.35 

0.2 

24.35 

0,2 

25,00 

-0.5 

44 

TUTT 

17.00 

22.70 

-5,7 

21,65 

-4.6 

21.65 

-4.6 

21,80 

—4 • fi 

45 

VALL 

24,70 

23.49 

1.2 

23.31 

1.4 

23,31 

1,4 

23,60 

1.1 

AVERAGE ABSo OIFF 
MSE 

• 

9,88 

2.54 

8.17 

2.29 

h-. 

• 

CO 

2,29 

8.59 

2.35 



Appendix C. Contour Maps 

Table C . 1 contains a description of the contour maps contained 
in the appendix and Table C . 2 contains the ranges used for each 
symbol in the maps. 


Figure 

Description 

C-1 

Method 10 applied to 51 data points, 196-2 

C-2 

Same, 1963 (48 points) 

C-3 

Same, 1964 (44 points) 

C-4 

Same, 1965 (52 points) 

C-5 

Same, 1966 (52 points) 

C-6 

Method 10 applied to 7 data points, 1962 

C-7 

Method 8 applied to 7 data points, 1963 

C-8 

Method 10 applied to 7 data points, 1964 

C-9 

Method 9 applied to 7 data points, 1965 

C-10 

Method 10 applied to 7 data-points, 1966 

C-11 

Method 8 (‘7 point data) after Step 2, 1963 

C-12 

Method 3 applied to 7 data points, 1963 


Table C.l Description of Contour Maps 


0 

rr 

less 

than 15 

1 

= 

15.0 

“ 17.5 

2 

= 

17.5 

- 20.0 

3 

= 

20.0 

-22.5 

4 

= 

22.5 

-25.0 

5 

- 

25.0 

- 27.5 

6 

= 

27.5 

- 30.0 

7 


greater than 30 


Table C.2 Ranges (bu./acre) for 

map symbols 
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+ 666b66666665t)6fc666fri 
I666666 6fcb6t>bf56bfcbob 
l6666666bfa66tj5666obb6 
+66666666666666666666 
I 66666660666666666666 


NOHTh DAKOTA DATA l9o2 : MEf-^OD '0 

1111 mill 1 111 ii'Tii \ 111 'iiiiiiiiin ti 't '( 1 mill jiiiiiiii 6666A66666666&6666666666666666066666666666+ 
n n uTi n n j nn'm nyiimiiim r''T rmimmn'nmn 666 fi 6666 fef>ftD 6 f, 6666 ti 666 fi 666666 fi 6 t. 666666656 A^.i 
777777 /^7T7777777^7?777/777 ^7 ^7077 / ^777777777777777 ^s6o666fa666hri66666n 666666666666666666661 

^ 77777 777 ?7777 7 / 777 77 777 7 7 77 777 7 7 7 7 7 777777777 7777 777 ^666 6t 6666666666 7)66 65555*5 6666666666666666666* 
777 71777 777777777 177777777 177 -min 7 7 171777777777777 6666666666666666666 55S555 66666666666666666661 


I666666bo666666666bb6 7 7 7777 77 7 777 7 7777 17 7777 777 7 7 7 7777 777 7 777777777777 66666666666666666666b 55*555 666666666666666666661 
*6666666666666066666 777777777777777777 17177 77777 7777 777777 17777 77777 6666666666666666666666666 666666666666666666666 + 
I 666666b66666 666666o 7777 77777777777777 777 777777777 7 77 7777 77 77777777 666 6666 6666666666666666666666666666666666666666666666 I 
I6666666666666666eb 7 777 7 7777 7777 77777 7 7 777 77V 7777 777 777777 7 77 77 7 7? 66666666e6666666666b6666666666666666666666666666666666 I 
*6 66666666 6666660,66 7 777 77777 7777777 777 7 7777 77 7 17777 777 7 777 177 77777 66666666b666666666666666666666666666666666666666666666* 
I6666h66 6666666666 177777 777777777 17777 ll nil 717777 17 17 77 1777777777 666b666667)666666b666666666b666666666666666666666666666l 
1666666666666666606 777777777^77777777777177777777777777777 rmTn 66660666666666666666666666666666666666666666666666666661 
+666666666666660066 7777777777 777 7 77777 7 777 77 7777 777 7 777 777^/7 77 ‘•6666660666666666666666606666666666666666666666666666666+ 
1666666666 666666666 77777 77777777 777777 / 7? 7777777 77 7 7777777/ 777 666/1666666666666666666666666666666666666666666666666666666! 
166666 666 660666666 66 7777777777777777777 177777777777777777 HI 6666666b66666666666666666666b6666666666666666666666666666 66 1 
+ 666666660666666666666 77777777777777777177777777777 17777 7 n 666606666666666666666666666666666666666666666666666666666666 + 
166666666666666666660666 7777777 777 77 1 17177 777 7777777777 66666 666666666 666 66666 6666666606666666666666666666666666666661 
166666666666666666666666666 7777777777 (7 1777 777 7777777 60666666 666666666666666666666666666666666666666666666666666661 
+6666666666666666660666666666 777777777/77777777777 6656066666 777 666666665666666666666 666 666666666666666606666666+ 
I6666c.66666666&66666666b66666666 77777 7 7 i 777 77777 6666666666666 7 6666666666666666666066666 77 6666660666666666666666 1 
1666666666666666656666666666666666 111 1777777 66666666606666666 66666666666666666666606666 77777 6666666666660666666661 
+66666666666666666666666066666066666 77/7777 b66bb6666666b66666666666b6666f 66666666666666666 777777 666666666666666666666* 
I6666666666665666b6666b6&666666666666 777?7 66666666606666606666666666666666666666666666666 77 66666606666666666666661 
166666666606666666656666606666 66666666 ^■)’77 666666666666666 d666666666A66666o 66‘*'666666666 77777 6666666666666666666b66o66 1 
+ 66666666 6666666666666-6666666666666566 '/ 7 7 66 b66666o66 6666o66b66666666b6b6 660 66666666666 77 77777 666660666666666666666 6666 ♦ 
I 6666666066660666fco666o6o6&66eo6666666 666 6666066 6 6066b5o6o6606666 6 6666660666 66666666 7777777 666666666.6666000666666066 ! 
I 6 0666 600 666 6606006o6o66b&666o6b666o66t'66o60600D 00660066606 060666 666 6 6666 666.666666066666 777777 6666666666666666666066O666I 
+ 66666 060 6666b66b6bo6D66666666666o6666o6b666c0666o66666 6606 000666 66666066 0666 6666666 666 777777 666666666666666666666666066* 
I 66666bb66666666666b66o66666600bo666666o6666666666666666660b00666666666666666fi666666666 777 66666666666666666666666666661 
I 66666666666 6666606666or)oo66 56006606666ob 0666666606 6666 66600666666666 66666666 666 0666666 6666666666666660666666666666 I 
+660/06666666606666666666666666666 6666 060666666606 66 56 666bo606666 666 6 60666 6 66 666666 666666 06 66 0666 666 66 66 6066666666 555+ 
! 6 666 6 66 666 606 66 06 66 60 66 66 66 6060 6 66 66 60 060 66 6066606600 60 6060 60666666 6666 66666666666 6666666 6606666666666 6 066666 55555551 
I 6066b60oo0666o6666b6666b 6066666666 6666 6066666 60666 6 6 0666 606 6 00.60666666606666666666666666 555S55555555I 
+5S5S55555555555 bo6666666666666&666fco0o66o6606o666o 066 600666 666 6 6666 06666666 666 66666 6 60666666660 555555555555555555* 
IS555S5S5S5Sbs555S5 6006666666666666 6b600000666666660o 66666O66OO6666666666666666666O06 555555555550555555555551 
15555555555555555555555 66666666666660 666666066666666606O 55 6666606666666606666666666 5'"'5555555555555555555555555I 
+5555555555555555555505555 6666666666O66666&6O0O 666 66666605 55 555555555555555555555555555555555* 
15555555555555.555555505555555 66666060 0066666 066 666 6660660 5555555555555555*^5555555555555555555555555555555 o5555555555l 
! 5555555555555555550505055555555 66660o0o660666660b666o0b 55555555S55555555‘i5S5555555S5S5 555S555555555S5 4^) I 
♦555505555555055555305555555555555 6666o6A6o6fc660656b0666 5o55'55555o55555555S555o555555555555555555555555 444444444444444+ 
1 555555 5o555S55b5655S5555555o555555 06 oo6a 6 666 666666600 555555555*^55555555555555555555555555555555555 444444444444444441 
1555555 555555555555555555555555 6006606666066666 5555o55555=^55o55b55553555555555555C55555S555 44444444444444444444 1 
+5 4444+ 555bS5555o555S5555555S 666666660 55555o55o555553565555S55555555555555S5555 44444444444444444444444+ 
I 44644446444<»44 5b5555oo555o55555555boSo5 55555b505555355555o55o55555555555555555 4444444444444444444444444441 
l4 4444 44/r‘. 4444 4‘*4 5b555b555555555555555b55ob5555555555b55b555555555555555555555555555 44444444444444444444444 444444 44 J 
+444/14444 3 44444444 55555555555555556555555555555555555555055555555555555555555555 444444444444444444444444444444444 > 
14444444 333 <^44444 44 55o 55555 53 505555555 55555 55555 55 55555355555 5555 555 555555 4444444444 4444444444444 33 1 
14 4444 444 3 44 4 444 4 ‘*444 555555555355655565555550550 555 ‘j53b3S5555*i5555o55555o 4444444444^4444444444 3333333333333333! 
+ 444444444 4 44444 44 44444 4 5b5o555o555b035 555 5555550555 5555 o5555S 5555** 55555 4444444444444444444 3.3333333333333333333333 + 
144444444444444444 4444444 44 555555 555^3 !>5555 5 555555 5555555 oObS 55 55 55 5555 *4444 4 4 44 444444444 44 333333335333333333333333331 
14 444444 444 44444444 444444444 4 5555555555’,5555S5bb55555 5555 055555 55555 444 444444’t4444 44 4A44 4 333.3333J3333333333J33333333! 
+444444444444444444444444444444 5555555bo<;S5Sobb5b 5555055555555 4444444444444444444444 3333333333 33333333+ 
14444444444444444444444444444444 55B5!50b5555555o 66 06 55505555555 <,4444444444444444444444 33333333333 ??2?222222 3! 
1444444 4444444444444*444444444444 555^^^3500555555 6 06 5b0o55555 *4*44444444444444444444 3333.3333333 2?222222222222?222 I 
+44 3 4444444444*4*44444444*4444 555555555555555 5 555bo5555 444*4444444**444***4444*4 33333333333 ??22222222222222222+ 

0 1 2 3 4 5 6 7 






■ + --— — + _— ~_ + — 4. o 


10 


11 


12 





NOKTn DAKOTA DATA lybj : Mt I MOD 10 

9 +55555555555555b555bbb55b355SbSS5b55S?ibbbb55E!i55bbbb'-i‘35&55b3555S‘i'=;'^=ibSSb5‘j55iS‘5S‘=:=;'^55b5‘>bSSbS5bbS555b55S5‘55S5SS555 66666666 > 
I55bb5bbbbbbbbbbbSbbbbnbbbbbb jbbbbhbb'^bbbbbbbbbbbbbbbbbbbbbbbbbbbbSSbbbbbbbbbbbbbSbbbbbbbbbbbbbSb SbbbbbbbbbbbbSb 666666661 
I5555bbSbbSb5b5bSbbbbbbSbbS5bbbb5b5b55bbb6bbb5bSbb5bbbBbbbbbb5b5b5£Sb55b6bbbbSb5b55b5b55bbb5bb5 A bb55S5bb5b555 66666661 
+5B555B5b5b5b555bbbbbb55b5‘i55b3b5bb5b6bbb5555b5bbbbbbbbbbbbb5Sb5'5BbS5?b5SS55b5B5S5555b5b565555 A4A444 bbS555bbb5555 666666+ 
IBSBbBbbbbbbbbbbSbbbbbbbbbbbbSbSSbbbSbbbbSSSSbbbbbbbbbbBbbbbBbbbbbbSbbbSbbSSbbSSbSbbSbSbbBbbS 44444444 5565555555555 666661 
I55555555555b6555bb555b5bbb555b55b5555bb5bb5b55bbb5bbb5bbbbb5b5555555555bb55S5555555b555555555 444444 555555555555555 66661 
8 +S555555b5555555b55555b5b555555b555555bbb5S5555555r35555S5b5b55b555555555555b5555555555555555555 55555555555555555 666+ 

IS55555555555b55555bb5bbb55S5b55555S555bbS55Sb55 55b555555555b55555S55555555bbb5b55555b55555S565555556555555555S 661 

I b5b555555555555555555bbb 44444,-^44 555bb55555S55555555555bb555555S55555555555555555555555b5b55 1 

+444444444444444+ 555555555 444+4444444444444444 555555555555555555555555555555555555 555555555555555555555+ 

14444+4+4444444444444 555 4+444,444444444444444444+4+44444444 55555555555555555555555555555555 6 555555555555555'55 I 

1444444+44444+444444+44 5555 44444444+4+44444444444444444444444444 55S55b55555555b5b55555555555555 6666 555555555555555551 

7 *4444444444444444444444 5555 +4444444++++44444444 444444444+4+444+44 55555555555555555555555555555 666 565555555555555555+ 

144444444+44444+4+444+4 fe 444444++4444+4444444444444444+++4444444444 555555555555555555555555555 55555555555555555551 

144444 44444444+44444444 44444444444+4++4 44 4 4444 4++ 444+ 4+4444 44 444 4 44 55555555555555555555555555555555555555555555555551 

♦4444 55 4444+4444+44444+444444444444+4+44+444444444+44+444+444444444444+ 55555555555555555555555555555555555555555555555+ 

14444 555 444444444444444444444444+44+4+444444444+4+4444444+44444+444444444 655555555S5555555555555555555555555555555S5555I 
14444 5 44444 44 44 4444444+44 +4+44444444^444444444444+44444444444 4 +444444444 5555555555555-355555555555555555555555555555551 

6 *444444 444444+44444+444444444444444++4+4444‘t44+t44+4444444++444444+4+4+44444 5555555555 555555555555555555555555555555+ 

14444444+4444444 4444 444+444 44 444 444444+4+444 4444444+ +44444 4<t44+ 4 4444+4+4+4 44444+ 5555555555555555355555555555555555551 

144444+4+444+4 J 44+44 +4 44 444 4444 44 44+4444444444+44444444 4+4+44+ 444444+44444444444 44 55555555555555555555555555555555551 

♦4444444+44444 3J3J 4444444444444444+44+444444444444444444+4444+44444444444444444+44444 55555556555555555555555555555555* 

14444444+44 3333333 4444444+444444+4444444444444444+444+4444+44444444444444444 5555 5555555555 5555555555 65555SI 

I 333333333333333333333 4+4444 +4+44444444444444444444444444444 5555555555555555555555555555551 

5 + 333333333333333333i333J33333J33335333 444 333333333333333 44444+4+4444444444 44444444 55 55=^5555555555555555+ 

I333333333333333333333333J3333 ;33 33333J 3 333333^333333333-33333 44444 + 444444 3 44444 + 4 5 444444444 556555555655555551 

i 33333333333333333533 o33j233 J333333333J3333333333J333333333-333333331 +4+444 3333 4+4444 +4444+444444 55555555 I 

+ 333333333333333333j3333333333333333.333.3333333333j33333333j3333333.33333 4 33333333 44444 444 + 4 + 4444444 5656 44444444 + 

133333333333333333.333333333333333333333J-33333333333333333333333333'^333333 333333333333 4444444444444444 5555 444444444441 

1 3333333333 33333333333333333333 33333333333 3333 33333333333 33333 3333. 333 3 333 333 3 333333333333 444 444444 444444 5 +44444 444444 I 

4 V 3 33 333 33333333333333333333333333333 33333 ^ 3333 3333 333 333 3333 3 3333 3 33 3 333 3333 3 3333333333333 +444 44444 44444 4444 + 44 4 4 + 444 44 + 

I 333333333333333333333333333 333333 33333333333333.333 3 3333333333333 333333 33333333333333333333 44444 444444 4444444444444444 41 

I 3333333^3333333333333333333.23333 2 3333 3333333 3333^3 333.-? 133 333 333 331 33333333333-^3333 44444444444444^444444444441 

♦ 3333333333 333333333333333.533333 2222?2222222 3333333333333333333333333333333333 33333-^3333-13 44444444444444444444444* 

1 33333333.233JD3333333J33333333 22222222 222222222 33 3 33333 333 3 333333 J33333 3333 3333333333333.3 44444444444444444444 1 

13333333333333333333333333333 22222222222222222222222222? 333333533333333333333333333333333333333 4444444 444 + 4 44 4 444 1 

3 +3333333333333333 3333333333 2222222??2+222222222222222?2222?2? 333333333333333333333.3 44444444444444444* 

13333333333333333333.333333 2?2222?222?2+2222222222222222222+22?22??222H?222?2?2??2 33333333333333.3333333 444444444444444+ i 

1333333333 333233333333333 22222?22222?2 + 2?2222222222?22?222 + 22?222?22222222222222?22 3333333 333333333333 +444444444 + 4 + 44 I 

+33333333333333333333333 2222?2??2222?222?2c2222H2222222222222?????22?2222222?222?2222? 33333333333333333 4444+ ♦ 

1333333333333333333333 2222222222222222?2222222 22222222+22?22?22222222222?2222?222?22? 333333.3333333333 3333333331 

I 33J33333333 22222222222222222+222222222 22?22?2222?? v?2??22222?22?2?22222?2222???2 333.3333333333333333333333331 

2 +22222 2222+22222?2222?222??H222222222222222222222222222??2?22222222222??22?222?2??22 3333333333333333333333333* 

1 2???22 22222222222 222222222?222222222?222??2?222?2222222222222?22???2?22?22?2?2?22?22222??2222??2 33333333333333333333331 

1 2??2222222222222222222222222222222222++2222222222222222222+22?222??2?22222?2?2?2?2222222?2??22??2?? 333333.33333333333331 

♦ 222222222 22222222222222222222222222?22222?22 2222222 2222222 +2222 ???22?222?2?2?2?2?222??2??22222?22?2?22 3333333333333333.3 + 

I22222222222222222222222222222222222 2?22?H2?2?22?2??2?2?22?2 33333333333.33331 

1 22 ?22 2 P22222 2222222222222222222222 1 1 1 1 1 1 1 1 1 1 M 111 1 11 U 1 1 1 1 1 1 1 in 1 1 11 1 1 11 11 1 1 11 1 22?2?22?222??2222??22?2?2 33333 3333333331 

1 +222222222222222222222?22??2222222 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 U 1 1 i 1 1 1 1 1 1 1 1 11 1 1 11 11 1 1 1 i 1 11 1 1 222222222222222222H22??? 33333333333 + 

I222222222222222222222222222222222 1111111111111111111111111111111111111111111111111 222222222222??2??22??22?22 33331 

1222222222 2222222222222222222222 111111111111111111111111111111)1)111111111111111111 22222222?22???2222?22?222222222 ' I 

+2222222222222222222222222222222 111111111111111111111111111111111111111111111111111111 ??22222222?2222222 222222222222222* 
122222222222222222222222222222 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 11 1 11 1 1 1 1 1 1 1 1 1 1 1 222222?222?22222 222222222222221 
1222222222222222222222222222 11111111111 111111111111111111 11111 noOOOOOO lUlllllUllll 2??2?.22?2?22222 2222222222222221 

C +2222222222222222222222222 1 1 1 1 1 11 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 i 1 U 1 1 1 1 1 OOOOOPOOOOOn llllllllllllll P22222222222222222222222222222 + 

e + „ — .,* — + + — _-*-~ — — * » + __ — + . 

0 ' 2.3 4 5 6 7 8 9 10 11 12 sD 



NOKTU DAKOTA DATA I'ibA ! HKTriOO 

9 ♦^4.4i,4A4A44AAA4AAA44<* BBbb'SSbSSS'jbbS 6666666b666fefc6b6666666tJo&6664f'6h6fi6A66fe66666ft66h6666A66Af!A6fo6A66f>666A66 77 7.7777777 777 + 
l4444<*4444^^*+44-444444 S5b5b5bbbbbb5b 66t>6('>6660ih6666b6C66^>6feDb6^)6bAA66^^^if?boft66AftA6AA6&b^C5f>(*'6c>66AAA67%67''6666f>6A 7777 777777 7771 
I 444444444444444444444 S5bS5bb5555bb bbob 6666U66 066b 6t3&6{>66o6 6 66 6 6666 6666666 b(Sf-66Afci666666666666fi6 666-66666666 7 77 7777777771 
+4444444444444444444444 b5555bb5555 66666666666o66o6666b6bD6666666666666666666666666fi666666666 5 6666666666 77777777777+ 

I 444444444(t44444A44A4444 5555555 6666 666666666666 66666666666666666666666666666 5555 6666666666 77777777771 

13 4444444444444^444444444444 555555 6 55555555b 66666666 555 46666666666666666666666666666 555 66666666666 77777777771 

8 +33 4444A4444444444444444444 55555555 555555555555555 5555 666666666666666666666666666666 6666666666666 777777777+ 

1333 44444444444444444444444 5555555555b56555S5555555555555555555 6666666666b66666666666666666666666666666666666 7777777771 
13333 44444444444444444444444 5b55b5555b5B555b555b555b55555b555555 6666666666666666666666666b6666666666666666666 77777777! 

+33333 4444444444444444444444 53555555S5bS55bb555b5b5555555b55555555 6&6666666666666666666b6666666666666666666666 77777777+ 

1333333 4444444444444444444444 5555555b555b5b55b5555555bbbb5555555b5 666666666666666666666666666666666666666666 77777771 

13333333 444 444444 44444444444444 5555^>535555555b5b5S55b555ba555555555555 666666666666666666666666666,66666666666666 7777 771 

7 +33333333 444444444444444 44444444 55555555555 55S555Sbb55555555555555 666666666666666666666666666666666666666 77777+ 

1333333333 44444444444444444444444444 5bbS555555 44 5555555555555555555555555 666666666666666666666666666b6666666666 77771 

13333333333 44444<*4444444444444+44A 556555555 44 555555555555555555555555555 6666666666666666666666666666666666666 77l 

+333333 333333 444444444444444444444 5555555 44 56555Sbb55.555555555S555555555 6666666666666666666666666666666666666 + 

133333 44 35333333 44444444444444444444 5 4444 555555555555555555555555555555 6666666666b666666666666666666666666666I 

1333333 333333333333 44444444444444444444 44444444 555555555555555555555555555555 6666666666666666666‘='6666h66666666666l 

6 + 3 3 333 3 J5 3333333333 3333 4444444v44<+4444444444444 44444 55bb5555555 55555555555555555 66666b66666666666666666666666666666+ 

1333333333333333 33333333 4444444444444444444^444444444 5 555555 5 55‘$56555b 5555555555555 6666666666666b66666666666666666666l 

I3333333333J33 ? 333333333 4A4444444444444444444444444 55555555555555355555555555555 6666666666666666666666666666666661 

+3333333333333 222 33333333333 4444444444444444444444444444 5555555555555555555555555555 66666666666666666666666666666666+ 

1 333333333333 3333333333333 4444444444444444444444444444 55555555555S5555555555555555 66666666666666666666666666666661 

122 333333333333333333333 444444444444444444444444‘>4«* 555555555555555555555555555 6h66666666666666666666666666666 1 

5 +22222222222 33333333333333333333 444*+<t4444444444444444444444 555555bS55555555555555SS5 66666666666666666666666666666666+ 

122222222222222 333333333333333 333 4444444444444444444444-4444 55555b55555b555555555555‘=^ 666666 6666666666666666666661 

1 22222222222222222 3333333333333333"5 4444444-»4444444444 + <m44 55555555555555555555555^ 666 5'='.55 6666666666666666661 

'22222222222222222222 3 3333333333333333 3 444444444444+4444 5555555555*^555555555555 5556555555 66666666666666666+ . 

122222222222222222222222 3333333333333333333333 44444444444444 555555555555S5555555S555555555S555 666 6666661 

12222222222222222222222222 333333333333^33333333333 33 44444444444444 555555555555555555555555555 5555555 I 

4 +22222222222222222222222222 333333333333J33333333J3333333 +4444444444444444444 5555555555556555555555555555555555555555+ 

1 22222 222? 222222222222222222 3333333333 23333333332 335333333 +4 4 4 44 44+444 44 44 44 55555555555555555555555555555655555551 

122222222222 22222222222222 5333333332333333333333333323333333 2+44444444444444444 5555555555556555555555555556555551 

+222222222222 2222222222222222 33333333333333333333333353333 3333 444444444444444444444 55555555555555565555555555+ 

12222222222222222222222222222222 33 33333333335333333 2 33.+33 4444444*444+44444444444444 555555555555^55655551 

122?22222222222222222222222222222222 2222 33333 333333333333 22 333333 +4444444444444444444444444444 555555555555551 

3 +22222222222222222222c2222?2222?2222?22222 3333333333333333 33 3333333333 444444444444444444444444444444444 55555555+ 

1 222 2222222222222* 222422222222222?2222242h 33333333333333333333,3 333 333333' 4 44 4 44444 4 4 + 444 444444 444 4444444444 444 551 

1 22 22222222 2222222? 2 22 2 2 22 2 22 2222 2222 2*2222 2 33333333333333333333 3333333333 4444444 44444444444 4444444444 44444 44 4 444 444 I 

+2222222222222222222222222222222222222222222222 33333333333333333333333333333 44 4444444444444444444444444444444444444+ 

12222222222222222222222*222222222222222422222222 33 33333333333333333 3333 44444444444444444444444444444441 

1 22222222222222222222*222*222222?2222?*2222222222 22222222*22 333333333333333333333733 44444444444444444444441 

2 ♦222222222222222222222222222?2222?22??2*222222222222222222?*222?2?2 333333333333333333333333333333 4444444444444444+ 

12222222222222?2?2222222222222222222?2222?2222222222222222222222???22? 3333333333333333333333333333333333 444444444441 

1 2222222*2 222222222222222*2222?22222??2*222222cH22222?22222*222?2???2?22 33333333333333333333333.3333333333333 444441 

♦2222222222222222*222*222222222222222?*22222222222222222H22*222222??2?22222 33333333333333333333.3333333333333333333 + 

12222222222222222222222222222*22222 23222222222222222222222 2??2?2222?222 333333.33333333333333333333333333333333331 

1222222222*22222222222222222222222 lull 2222222222222?2222*22222??22???2*2?????2? 33333333 3333333333333333333333331 

1 +22222222222222222222222222222222 lllliil 222*222*2222*222*22?22??22??2?222*??2???2?2?2 ?? 3333333333333333333333+ 

1 22*22222222222*2222222222222 UlllliUl 2222222222222222222??22??2222?2???222222?2??2222?????? 333333333333333331 

111 222222222222222222 111111111111111 222?222*22222??????222222???2?22?2?22?22?22?22?22?2?2 333333333331 

♦ 1111111111 lllllllllllinililllUl 22??22?2222222?22222E?2?22?2222?22222222?2222??2 3333333 + 

mil 11111111111111111111111111 n 1111) llllllllllliimiimilll ??2?22?2222??2???2222*??2222?2??22??????22??22?22 331 

iiiiiiiiiiiiiiiiiiiiiiiiiiiimmiiiimiiiiiiiuniiiiiiiiiinn i 

0 +imillilimimiuilllllllllllllllllliimilllimilimilinnim 222?222222?2?22?2??2222?????2?222?2?22222222222? + 

0 1 2 3 + 5 6 7 fi 9 10 1112^30 



NOUTH OAKOIA data lyob ! MtThOD I'l 

9 ♦5555555)b55b 6666666b6t>66b6t3b&6666f>f)66tit)&t>(36 666hfat)6666 666boo66666f.(=>66 777/77 77 7777777 77 777777 777 777777777777777 77777777777* 
I5565b55b55b 66bb6b6566b666bb66&6bbftfiobbb6hbbbbbbbbbbbbb6bfa6 6b666A6b ninTmin^lTWllTn mmnm'miTnllinnmil 
Ib55b555b55555 6bb66b&6bb66666666b666bb6bb66666b66b6b&6666o66bb6f.Af,6Af> inilinmn 77777777/7 77 77777777 7 77777/77777777771 

+ 555b55bb555bb55 66b6b666666b66666666bbb8666bbo66666b66f36bb66b66f>6bb666 Tm mnlll 11111111 timitnTll mni'niminil *■ 

ISSSbSbbbSSbSbSBSb 66b6ebo66bb667>6666bofat3 666fi6666b6b6666bbbb6b66 66f>66o66 7777777777 7 77777777777777777777777777777777777771 
I5555S5b555b35555b5 6666666666666666bfibt3faft(Sb666b6b6b66666bb66666r)A666bbbbb 1111111 111 11 1111111111111111 111111111111111 1 1ll \ 
a ♦55bb5555'i5bbbSbbbbS 6666bb66n6666666fc>ob66b6bbfa6b6b66fa6fabbb6bbb6f)bhb6bb6bb /7 7 777 777 77777777777777777 77777777 77 77 777 777 77* 
15555555555b5b5Sbbb5bb 6bb6666666D6666o66b666666b6b666666bb66h66bb6fa66666bft 7777777777777777777777777777777777777777777771 

I555b655555b5bbSb55bbbb5 66666b666666b6666666b6b666bb666bfa6fa66bh'6666b6666b6 77777777777777777777777777/77777777777777771 

+ bbb5355bbbS55b5b55bb555 6666666obA6666fa 6<S 6f)b66666666fS6Ab 777777777777777777777777777777777777777777* 

IA4A b5b555555Bb'555b5S5b55555b bbSBSbS bbbbbSRH 66bf)b66666f.6A6A 111111111111111111111111111111111111111111 

I444A4A 55bb55b5bbbb55bb5bbbSb5b5b'55b5bbb5535bbb5Sbbb55bbbb5fi55 ftb6f,66666bbAbbb 11111111111111111111111111111111111111111 
7 b55bbbbbsbSbbbBbbbb5bbb55bbbSb35b5S5bbbbbS535Sbab55bS 66(S66666b6bftb66 777777777777777777777777777777777777777 + 

li*46.4 444 444<*4 bb5b5bbbbbbbSb5bb3bb55'^5bbbS5555bbbSb555b5bbb5bPb5'^ 66b6b6bbb6666666 111111111111111111111111111111111111111 
144444444444444 bb=.bb5bbbbbb5bbbbb3b5bbbb5b 5b55bbb5bb5556Sbb5bbSSS 6A6666666fS666666 1111111111111111111111 1111111111111111 
+ 444444444444444 5b5SbSbb555bbbbbb55bbbbb555bbbbb56Sbbb5bbbbbbbbbb 6b666b6bbfib 666 1111111111111111111111111111111111111 * 

144444 4444444444 bbSbbbSbbSbbbbbbSbbbSbbbbSbbbbbbbbPPbbbbbbPbpiSSS 6bb666666 77 666 Hill 111111111111111111111.11111111111 

14444444444444444444 5555b5b5b5Sb55bPbbbbb5bSbb5b5!55b555bbb5b5b5b5PS 66bb6666 6666 1111111111111111111111111111111111111 

e +444444444444444444^44 5b55bSbbbb5Sb'4bbSbbbbbbbbSbbbb555bbbb5bb55P55Sb 66666666666666 11711111111111111111111111111111111* 

I 444 4 4 44 44 4444 444 4444 4 4u b5bB5bbbbbbbbb'jbb5S5Sbb55bbb';bbijb55bb5^‘4n5r>5b bh666 665666b66 117711117 17111171777111 11 1117711 111 

1444 4 44 4 44444 44444 4444 44444 555bbbSbSbbbbbS5b55bbbbbbbS5bbbbbbbPSbb555bb 666 6666666666 17 7111 111111111111711111171111111 1 
+4444444444444444444444444444 b5bb6bbbbb6b5555bbbbbbbb55bbb5Sbbbbb‘>5b55b5b 666666666666 1111111111111711711711111111111 + 

I 444 44 44 444444 +4444444 *44444444 bb5bP553 3bbbSb5bbbS5555S5'jbbb5bbbbS5Pb5S555b 666666666666 777777777777777 777777777 7771 

I 44/>4444444444 44 44*44444444444 444 SSSbBbbbbbbSSSbbbSbbbbb&bbbSSSbPSbbbSSSbSbbPS 666666666666 117171711111111171117111 1 

5 * *444444444444444444444444444444 55Sbbb55bbb55bb5b5b555bbbb55b5S55bP53555b55bb555 6666666666666 1117111711111711111111 * 

133 444444444444444+44444444A444444 BSbbbbbbbSbS bbbb5bS5S5bb'^555b55555555b55 66666666666666 111111111111111111111 

1333 444444444444444444444444^444444 *4444444 bbSbbbbSSbbSSbbbbbBPbbbSPb 666666666666666 77777777777777777771 

+33333 4444444444444*44444*44ii4444444444*444444444**4444444 bbbSSbSbSbbbbbbbBSbbSbSb 6666666666666666 777777777777777777* 

13333333 44444444444*44444444**444444444444444444444444444444 BSPbSbbBbbSbbbbSBbSSbb 6666666b6666666 7777777777777777771 

1333333333 444444444*4444444444444444444*44444444444444444444444 5<5b5bSb55b55565555B 666666666666666 777777777777777/771 

4 +33333333333 444444444 444*4 44*4 444444 4 444 4444 4 4 44*4*44 4444 4444 4444 5b55555655b6555555 66666666666666 111111111111 11111 * 

133333333333333 4444444444444444444444444444444444444A444444444444444 555S5S5b5b5555556B 6666666666666 7777771 

13333333333333333 4444444444444444444444444444444444444444444444444AA44 S5b55b5S5S56bPb5Sb5 6666666666666666666666 71 

+333333333333333333 444444444*44444444444444444444444444444444444444444 5555555555b5555b55b 6666666666666666666666666 + 

1333333333333333333333 44444*444444444 4444*4444*444*4444 444444444444 555555555555*'5b'55Sb 66666666666666666666666661 

133333333333333333333333 444444444444 33 4444444444*4444444 44*444444444444 5b5555555=^65Pb55b 666666666666666666666661 

3 +3333333333333333^333335333 4444444 44*444444*4444444*4444444*444*44444444 bbbbBSbbbbSbbSbbP 666666666666666666666+ 

I3333333333j33333j3j3j33333333333 3 444444444*4444444*44444444444*444444444* bbbSbbbbbbbbSbbSbb 666666666666666666661 

I 3333333J3333o5333333J33333333333333333333 444444444444444*44444444444444444444444 b5S55S55S5b56555SP 6666666666666666661 

+ 3 3333 3333 33333333333 J3J3JJ333T 333333 33 33333 4444*444 4 44*44 4 4 444 4 44444 4 44 44 44 4444 BPSbbbSSbPSPb'jbBPbS 66666666666666666 + 

1 333 33333333333 333 333 J333333333 33333333333333333 4444 444446444444444444444 444 4 4444 bBSCbbSbbSBbbbSBSbS 6666666666666661 

1 3333333333333333333333333333333333333333333333333.33 444444 4 44 4 44 4*44 44 4 444 4 44444 BbbSbSSbbbBBSbSSbSPb 6666666666661 

2 + 333333333 . 3333333333333333333333333333333,33333333333333333 4444444444444444444444* 6656b56b5Pb6=.56P6b6bP 666666666 + 

1 33333333333 333333333333 333.3 333.33333333 333 3333333.3333 35333333333 4444444444444444444444 SSbbbbibbSPSbbPbbSbPPSb 66666661 

I'3 333333333333 3j33J3JJ333333333J333333333 j 333 333 3 53 333333333 3333333 44 444 44 4 4444444 444444 bbbPbbSBSbbbPbSPSSbbbbS 6666! 

+ 33333333333333333333333333333'5 53333 3.33333333333333333333333333333333 4444 444 444 44444 444444 4 SSbbSbSbbSbSSPSbbSSSbSbb 6 + 

1 33333333333333333333333333333333333.3.333333333333333333333333333333333 44444444444444444444444 555665566665655555555655 1 

I 33333333333333333333333333333333333333333333333333333333333333333.3333 4444444444444*44444444*4 55565656b656SS5555S56b66 I 

1 +? 33333333333333 5333333.T3.333333 33o3:3 333333333333333 33 3333 333.3333333 4444 4444 44444 44444444444 56655565655555555555556 + 

12222222 3333 333 j33J H 3333333.333.3 33*>333.3333333333333333J33333.3.3?333333 44 4444 4 44444 4 4444444444 56555556555555555555551 

12222222222222 33 33 333333333333 33333333333333 3333 333333333j33?3, 3 3333. 3 333333 44444444444 4444 4444 555565656565556555565551 

+ 222222222222222 333333333333J 33.33333 3333333 333 j3333j333 3 3^33333 333 3 33.?33 3 333 3 44444444444 444 555S55S65556555S55555566 + 

122222222222222222 333 J33333J3333 33 33334333333333333333333333333 33333333 33333.33 3 44444444444'+ 55555555556555555555555551 

122222222222222222222 33333333333333 J33333333333333333333333333333 3333333333333 4444444444 55 555555555555555555551 

0 +222222222222222222222222 333333333333333433333333 22 3333333333333 4444444444 444 555656555565555555+ 

0 1 2 3 4 5 6 7 a 9 10 11 12^ 



NOUTH UA^O^rt data iVti6 i KtlMOO 

9 +4444'!i sb55bb5b5bb5SSb‘jb5b55{5‘5y:^bibtv3 ’bobtb35b5bbb3Sb‘j5b5’=;‘^'4b'j:)rs5b&=ibS‘7'5 <=' 666 b 6 b^bb^bbbbbbSSbSbb'^^bbbbSb'ib 666666 f> + 

I44444444«4444 bbbbb'ibSsobbsbobbbbbS b;ibbSb'-'3bbbbb:^bsbb5 = b JbSbh54bbb^‘^b‘'bb?ib‘ib=i f>4r.b6 b'’>=^bbbTb=ibS5bSbS5'=>S‘55b'^bS5 66 ftt>f.fil 
I44444444-4444444 bbbbSbbtobsbb'jbbSbb 6 b«,bbbbbbb’-jbb“ibbbbbib5bbb54'-.'=i5b«bbbbbb‘^bbb‘= 6666 bH5b‘- SSSbbbSbbbbbSbbb'^bSSbS'i 666 1 
+ 44444444444444A4 bbbbabsbbbbbbbbbbS 666 bbSbbbbbbSbbbhbbbbbbr)b5566bbSbbt)bbSbS‘''=b‘i 6666 S'^bb6bSbbbbb‘5b5‘5b‘i555b5b55bS555 6 * 

l4444.‘+444444-m4444 bSbbbbbbbSbbbbb 666 b bbr5bbbbbbbbb5bbbb3bbSbbb5bbbbbb5bSb5b55b5 666 bbbbSbSSS 5bb5555bbb'i55555555555 I 
I4444444<*44444444444 bobbobbSabbbbbb 6 bbbb5bbbb'jbbb!:bbbbbJbb'^br-‘'44bbbbbbb‘^b6b‘=:bbb b‘",'"ibbbbbbbb'ibbSS5'ibbSb?.b5b55S5Sbbbb I 

a +4444444444-^•*^44444444 bbbGbbbbbbbbsb bbbbbbb3bbbb53bbbb5bb3b5'5‘>b‘'‘'bb‘^bbbEb5b4‘>bbbbbbb5b5bbbbbbSbbS5bbb5bb'^b5bbb5S5SSr)F.bb?. * 
144444444444444444444 44 bbbbbt>bbbbbb?b6bbbbbSbbbbbbbbbbbbbsbbbbbbb';bbbbbb5'ib'^5ShbS55b''bS"SbF'bB'if;S'^5P95ab55SbbSbS55Bb55b5SI 
1444444444444444444444444 5SbS55b5b5bbD5SbS!3b5bbbbbbb5bbbbb5b55b'?b5bb’jb5bbbbS565455bb'i5bbbb5bS';bbb5‘i6b5bS5b5bb55555bb5b55 1 
+ 4444 444 4 444 4444 A4 4444 44444 b55b5b5b'-,bbb65b5bbb5b5bb6b555i> bbb=lbbS5‘F5bbbbbb'3bb5b5b55Sbbb?bbb5bbbb'5b5bbbbb5bbS55S555b55 + 

1444 4444 444 4444444444 4444444 4 bbbSSbbbbbbbbbbbbbbbbbb 6444 bFbb'^Sb55bb3S5S5b55b=155S‘^bbbbbb555bbbbb5bb5Sb5S5bb55b5b55I 

14444444444444444‘+44444444444444 b'i6bbb=,5bbbb 44‘r44444 44 bbS bS5 bbbbbbbbbbbEibbbSSbb bb';S‘ibb?‘iS‘vSbbb'^=155b555 55555 SI 

7 ♦ 44444*t444-*‘t4444*t444444‘.44444 bbsb'^br.bs m 4t4‘t4 -+444 ■+4 4 ij , 44 4 bbbb jbf-b^bb 55bb5''bb5‘-5“5b4'35'-4,t,c, t,Ktjc. r^c^t c,cs,r,4c>r;5c t,ct, 

1333333 44 44444444444‘t-+44444444 4.,,4 bbbr.bbbb 44 4 44-*444444 44 444444 b bSbSbb br.'?b5555bbbr.b-'^bb‘-' r>s5bb=;S‘=^5 55664 4r-,q4a5b55b55555 I 

I333333Jj 33 4 444 444-»44**4 4444 44444 4»»;+4„ bb'-Bi'bS 44<,44 4444444 4444444 4- 665^55 556=065 bbbb6c,t,t,b6bb5566b‘.6b55Sb656656655666ba‘' I 

*333333j3333 4444444444444444644444444 5bbb5s 4 44444 444444444 u 44< 4444 bSi 5555b656b66bb655556656566a6b56665655555555b55555 * 

13333333333333 444444444444444444444444 b 4446444444444444444444444 b bbb665S56bb5b555bbb55bbb555556555556bb5b55b5555 5 1 

1333333333333533 ^44444444644444444444 44444 4444444444 44444644-+444 444 4444 556555 =>5555555 = 6566565556665 65 6 656555 5556555555 1 

6 ■t-33333333 33j33333.i 44 4 44 44 4 ^ 444 44 44444 <-,i*4i,4<j<*t»4444444^, 44 ^ 44444444 , 66^^6655556665565^6 56556656g6♦ 
*3w3'j3JobjJ33o333j33 *+A4444‘+4444'+»+>-44444>f4‘+44444‘+>+4 >*+“’444*+4*>4444>*44 444444*+444 5556'+35b6555*=b666 6*^j66566666565555bb5555551 

133333j333ji53 3333.}33 44‘tHt4‘+4-44 4-»4^>TH444 44 44s44 + +4Ht44444'+-+444444444444i,<*4 4 4444 56555656 35 4 56555656 = 55555555555551 

33333333333333333 j 333 44<*444<*44-t4444 4**‘t4 4‘tt>*4‘+444‘+444-+44-+‘+444A-4 4 4T44‘»44H4444444,,444 5 = 55555 4 444 55 b 666655535 S 5555 = 66 <:;* 

I 33333333333333333333333 44444 444446444444444 4444 4444 44 4 46'+f4444'+ 444444** 4 444 44 444444 4444 44444 5555555555555555555 55 1 

I 333333333333333333333333 4444441 * 4444444444444 4 444444444444444444 ** 444444444444444444444444444 44 4 44 555555555555555555551 

5 -*33 3333 33 3333333 3333 J3333333 44444444**4t*4 44444444 '+4444444444444i444/* 444444444 444 444444444 44 444<*44444 65=555555555555555 + 

1 3333333333 3 3333 3333 33 3333 3 333 3 444 -' 44 *, 444 444 4444 444444 -+ 4444444444444444 4 ■’* 44444444444 4444 56=55555555=5551 

: 3.33333333333j333334 5333333333333.33 333Jo3333j33 4 444444444444444444444444444444444444444 55555555555551 

*33333333333333333333333j3 333-J33333333-iJ33333333333J3333333.5 3333 33,3 4444**4444 4444 444 44 44446444 4 44^44444 44 444 55555555 + 

1 33333333333333333333333333333333 A333333JJ 333333J33333333J3J3.5333333 4 44 44 '+44444444444-444 4 444444 44444 44 44 4 444 ( 1444(1 ^51 

133333333333333333333 333 J33,;33i 33333333 j3j333 3333 J33 333 3333,^5333 3 "533 4«*4444 44 4444 4 44 44 44 444444444 44 44 44 44444 44 44 4444444 i 

4 ♦ 33333333333333333333333333333333 j3 333333 j33. 53333 j, 33 333333 J 33533 3 333 3 3 4444 44 4 4444 44444 4444 44 4 444444 4444 44 44444444 4444 444 * 

I 333333 33. 3333333333333 3333333333 33333 33333333 333 33333 333333333333333333 . 3 44444*t4 4 44 4 44444 <* 44644444444444444 44444444444 4441 

1 333333333 3333333333333 j 33 . 33333333333333 o 33333333 j 3333 333333333336653-?,5333 444444444444 44 444444 4444 ** 4 44444 444 ** 44444444 <« 44 i 

♦333333333333333333333333 33333 133 333333+* j33333333333333333J3 333^.3333333333 444 4 444 44 - 44444 (, 44444444444 44 4444 444444444 444 + 

1 3333333333333333333333333333333333333 2 3333333333333333 2 333333333333333 44444444444444444**444444444444444444444444 1 

333333 333333.3333.J 2? 3333333533333333 444444 4 44444444444 44444 4444444 444i*44 444 4 I 

3 ^3j33333333433333w3J.i3343333.l33333Jj333.5 33333 .jJj333.^333.^333 5333333333333335333 4444*+444444(* 4 444444444444444^444444444 4 + 

I 3333333333jj3 + 3353333o3333 133 1.13o3 )3333.^3333333'+3333ojb333J333333333333*3.5333333333 44444444444444444444444444 4 4444444444 I 

I 33333333333333333 333 33 333333 3.1 33 33333J1333333333J3j33333o333333333.T333333 333333333 4 444444444444444 444444444446444444441 
+3333333j3333333333333333333333.1333j333 333333 333333333333333333333333333333333333 4444444644444444444444444444444444+ 

*'4333333333333333'433333333333333333333 2222 333 2 3333333j33j33.33333.33333."4 333333333333 44*+44444444444444444444444444444I 

I3p33p3333333333333jp:i333333333333’l3_ 222 22222 333333-5333333333333.33333 33333333.13 3.31 4444 44444 444444444 444444 4 1 

2 *J.5 33oo3333333333333.j433333334333.5 3 13333 3 2 (J 222 ■.''22 3333333333311.3.3 3333j333333333353333 333i333 44644444444444444444 + 

i3333333 333j.^43b3j.;j333j43j1334*/33'433333 2^222, c 2 2’ 42 333 3.3 3.133331333.= 3333333.1333 j333333 13 444444**4444 4 444441 

1333333 444** 33343j.13j33jj 33.13J 1333.13.3333333 H22222<r2222223322?2?? 333 1333333333.3333.13333333333333 33333-3 44H4444444444I 

*3333333 4444 33333333333 33333333333333 Jij'l 2222222222’2?222222222??? 3 3333333333.333.33.33333333333333333333333 44444444* 

I3333333J 44 333333333333 3 333333J33333j3 33 2222232222222223222??????5' .3333333333333333.333333.3333.3333333333.333333 41 

1 33333333 333333 3j3333333333J3333 3.3 33J33 2222 2c£- 2 222?..5 2 22 222 22???????2 3?3 3333 3333,33333333.33333'’333333333,3ri3333.3 33 333 3 I 



1222222222222222222 3333333333 

I 2222222222222222<;222 

*22222pH2222H2222222222222?222H22322?22222222222222222?2222222222P? 1 22?2??2222222??.?? 33-3333333.3333333333333333333333 + 

0 1 ^ 34567^9 10 11 12 


vO 

(» 



NOrtTH DAKOTA uAl A lS«fa2 : METHOD 


9 +77777 777777777777777777 777777777V77777V777 / /7T7777 /7777T77 /■777777777777777777/77777777r777777777777777777777777 666666666 + 
iiiniiiimi ni u-ninii niniiniiTiiTniiiii-i inirnnn 777777777777777777777777777777777777777777777777 6666666666 1 

UTfTnmii f 7 m'fn f rniTfim nrmiJ n-m rmnrfii'nmm rmiT/mm mymmnnmTmnnm 666666066666666 1 
*mimTtmyn n JIT ! ni HTn nmm nTmn irrm nn n nniTnm'niTmmrmmnnmTnm 6666666666666666666+ 

17777777 777777 7777 777 777777 7777 777 7777? minmmnTnTn immimmTnninn-nmimnn 6666666666666 6666666666 I 

17777777/777777777777777777777777777777 / 77777777777 n 7777 7 7 V7777777777777777777777777777777777 666666666666666666666666661 
8 nrimiiTt ifim nimmm 77777777777777777777777777777777777 666666666666666666666666 6666 + 

nmimmyiiTnirrm rnTmimim ninrniii n'j'imirmTnTrTnn-immnmmm 6666666666666666666666660666661 
I innirnm nmmmmmimi rrniT^i nnrnivn nnnmnjnmmmnnm 66666666666666666666666666666666 1 
+6666 TymiTmmmmmnmTry mmmTnmTn'n immimmmnmmn 666666666666666666666666 666666666 6+ 
166666666 7777 77 7777 77 7 V 7777 7 777 7777‘77 7 7 777 7 77 7 7 7 77 777 77 7 777777777 7 77777 7777777 777 7 6666666666666666666666666666666666661 

166066660666 77777 7 7777 77777777 77777777 777777 77 77777 77 7777 7 777 777777777777777777777 b6C-6666666666666666666666666666666066 I 

7 +666666666666 11 nVnmTr mTl'niml nymiimiyi nmmmimnimmmn 666666666666666666566666666666666666666 + 

I666666&0 656666 llJlTn H ll H mil 77 nilllT 7 imrniliniTn im 66666o66o66666666666666666666b66666666o6I 

16666 66660666666 / 77177777 7''7777777 777? 7 777 77 777 7 77 7 7 7777 77? 7 777 77777 7 7 77777177 6666666666666667-v66666666666666666666666666 1 
+ 66666600 66666666 7 77 7777777777777 17777 >777777 77777 77 nil! 1 / 7777 7777 77 7777 7777 66666666666666666666666b6666666666666666666* 
166666 666 666666666 7777 7 7 1777777 7777777 !7 7777 7 7 77 777 7 117 777 177 77777777777777 6666666666666o6666666666666666h6666666666666 1 

I 666666666066660666 7 177777777777777777 nimmilin 177777777777777777777 6 '^666606666566666666666666666666666666666566661 

6 +6666656bc66666o666b 177 7 17777 77 17 17771 H -i77 7 17 77 7 7 7777 77 777 7777 7 7 77 777 77 6666fi666666666666666666666666666666666666666 + • 

I666b&6666'j6 6566 666bo 77777 1 ', 777777 77777 7 7 177 77 7777 1 177777 1 ! 77777771777 66666 666666666666566666666666666666566666 55ISSI 

I666c,6566oat6666‘bC6bc6 71777777777777771717777777777 7777777 (777777777 666 666n666666fj6666666666666666666666f 66 '556655651 

+6666666666666666660666 7777 77777777 77? 7 77777777777 77777? 77? 77777777 666666666566666666d6565666666566666666 555555655555+ 

166666666556666666666666 77777777777777777777777777777777 11777717 6666666666666666666666666666666666666 5555555555555551 

1 66666 666666666666 60666666 777777777777717777777777 777 777 17777 666606666666666666666666666666666666 555555555555555551 

5 +666666006666660060066666066 777777777 nnmmn 7777777717 66666666666 66 6 666 66666666666666666666 55555555555555555555 + 

166066660666666000666006606666 77777777777771 1777777777 77 c66666666666666b5666666666666666667i66 56555555555555555555551 

I6666666o6666666666b6o6c6o6666666 777777777777177777 6666666666666666066666666660666666666 5555555555555555555555551 

+666666666666666606666666666666666666 77777777777 606666666666666666666606666666666666666 55555555555555555555555555+ 

1 6666660666666666666660666666666660666660 6666666066666666666666666666666666666666 55555555555555555555555555551 

I 66666 666 66666 6660 60666066666666666666666666 6666665666666 000606666666666666666666666666666 5555555555555555555555555555551 

4 + 6 66D6666666666b60b6666bb66666566o66660ob66666 66 6c 666666660066 6666 6666 660666 6666066666666 55555655555555555555555555555555+ 

I 6 6666 66 06666 6 be 6 6 66666 66 66 6 0&6 6606666 60 60666 6 666 6 6660666 fc 0 066 66 66 6666 66666 6 6 666 66666 66 ‘'555555555555555555555555555565551 

16666666666666666660666660 6606666060660666 666 660606660660 cbo 6666606666 6666666 666666666 55555 '^5 5555555555555555555555555 5 5 I 
+ 666666 666666 b666666666666660666666666666£666o666b6=6o66666b6o6666666666666666666666b 55555555555555555555555555555 4* 

1666 6 bo 6 6 666 6666 6666 0606666666606 06 66 6 06 066 6066 6666 66 66666 6 66666 55555556555555555555565555 44444441 

I 5555555555555555555 6666o666666666DbA66666666666666666b66666666666666666066666 55555555555555555555555 444444444441 ■ 

3 +5555555o555b55b5b56bo555 6066666606660606666660666666666666666666666666666066666 5555555555555555555555 444444444444444+ 

I 55355bb5b55b; 55bb55bb55555 66666666660666666 66666600666666066666666666666 555555555555555555555 444444444444444441 

I555b55b6b5b555bb5b5bbb555555 606066666066665 777777 666666066666666666666666666 555555555555555555555 44444444444444444441 
♦ 555o555b55o5S555b5bSbb5b5bob5 66666666066666 7777777 6666666666660666660666 555555555555555555555 44444444444444444444+ 

IS55555bb55bbb55556b5b55bbS5b55 66666f666666 777777777 b&66o66666666666666 5FB55S55555555555b55555' 4444444444444444444441 

I 555b55555555b555Sbbbbbbbob5b55 6606666666666 777777? 666606666666666666 5555555555555b55555S5555 444444444444444444444441 

2 + 55 b b- 5bo55ob555bobboobboo55b 055 6C666O0666666 7777 666666o666666666r')6 bbbbb^'ObSbbSboSSSbbSSbbSS 444444444444*+44444444444 + 

I 5o55o5bbobbbb5b O0bbbb5bbo555b5b 66666066606666 • 666666660666666666 5bbb5b55bbb5bb5bb55bb5bS5 4444444444444444444444A44 I 

I555b5Sb55bb5bbbbbbb55ob5b55b55b5 6666. oooo56666666 6 66 66666606666 6666 55556555555555555655555555 44444 4444444444+44444444441 
+555555o55S5Sb555bbb5b5b6b5555Sb5 66 66 0666 666066666 06b 666000666 666 6 65555555555555555555555555 444444444444444444444444444+ 
1555555b555ob55b5bbbi5o5b5b5bbb555S 6 6f'0ouo6666bo66666666 600066666 6 5b‘5bbbbb5bS5b55‘j55b5SS55b5 444444444444444444444444444 I 
I55bb5bbo5b5bS5o5o5bbbbb5b55555'j55 666606fr6666666666666666oO666t4 5 5555555 55555555555 555555 444444444444+4444444444444 31 

1 + 55bo36ob5bboo5bbbob5obb55b55 Joo5bb OAOoooOutoobbbObot oooCobboo 6t>Ao5535ob55o5i>5555b5b5555 ->4444444444444444444444444 333* 

I555b5ojo5bo'>b'oobo5ooo5bb6555bb55bbS 6b>»64o66o66c 00c 006660006 555555‘"o53bbbo55555565o5b5 4 44 444 4444 4 444 4+4444 4444 4 3333 31 

I555E5bbbl:3bob3boboo5oo55bbo5b6555b55 0646666 6066666066660 bbb555Bbb5 b5 555 55555555655 4444444 4444 444444 + 444444 33333331 

+b5S55bbb555Sb5bSbbb5b55bb5S5b555b5555b b65666&0666bo66 0bb5b5555b55b5b55b5.5b55SS5b 4444444444444444444444444 33333333.3* 

I 5555555555b5555bb55b55556555b55b5D555bbb 60606 b5bobb5b555b5555555555b555555 444444444444444444444444 33333333331 

1 55555555555 55 5b o55b5 3 d 555 55 555555 bSbSboba 5555 5b bbSbob 655555 55555555555 5555555 44444444444444+444444444 3333333333331 

0 +5S55555555b5bb555b5bo555b5555555b55555b5555555a5555b55 55b 5055555555555555 5555555 5 4444444444444444444444444 3333333333333+ 
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^tOHTH DAKOTA DATA 1963 ! METHOD )! 

+ 555 ?i 55 b 53 b 5 b 653 b = sbi 5 b 55 bbb 5 B 5 S'i> 5 'js 555 bb 3 =ibS 5 bbS 5 bb 6 bbSbSbb 655 S 5 b 5 !T'iiS 6666666666666(^6656666666666666666666666666666666666 + 
IS55b55bbbbb3355bbbs5bb5bb3b5bbb5bS55bbb3Sb55ribb5;3bbbb55bbbb5bbb5bb55 666666666A6666b666666666666A66666666666666666666666M 
IbSSbbbbbbbbbbSbbbSsbbSbbbbSbbSbbSbbbbbDbbbbSbbbbbSbbDbbbbbobbbBbPbbS 66 666 6666 666666666666666 Aft 66666666666666666666666666 I 
♦SSSbbbbbbSbbbbbbbbbbDbSbbSSSbbBbSbbbSbbbbSbSBbSbbbbbbbbbbbbbbSBbbbb 66666666666666666666666666666666666666666666666666666* 
I 555555 b 55 b 5 bbSS 5 bbbbb 555 b 5 Ebb 5 bbbbb 5 bbbb 55 b 5 bbb 5 b 555 b 555 b 5 b 55 bb‘ 5 '^B«; 
I 55555555555 b 5 b 55 Ebb 5 bb 5 bbb 555555 bb 55 Bbb 555 b 55 b 5 bS 55 bb 55555 b 555 bbb 5 b 
+ 55555 b 5 b 55 bbb 553 bb 5 bbb 5 bbbbb 35 b 5 bB 5 SbbSbb 5 b 55 b 5 b 5 bS 5 bbbbbbbb 5555 SSFi 
I 5 rb 555 bb 55 bb 3 5 bbS 55 bSbbb 5 bb 5 bbbbb 35 b 5 b 35 b 5555 bbbb 5 bbb 55 bbbbb' 55 b 5 bb'^ 
I 55555 Sbb 5555 D 55 bbbbb 5 bbbb 5 bbbb 55 bb 555 bb 3 bb 5 bbbb 5 bb 5 bbbE 5 bbbb 55 b 555 S 
+ 5555555 b 5 sbb 555 b 25 b 5 b 55 bbS 5 bS 5 b 5 b 5563 5 bb 5 5555 bSbbbbbbbbbbbbbbr^SSBbb 
I 555555 b 55 bbb 5 b 5 bb 555 b 55 b 5555 b 555 bb 5 b 5 bb 555 S 5 Sb 55555555555 bbb 5555 b 55 
I 5 S 55555 bS 5 bbb 5 Sbbbb 555 S 55 b 55 b 55 S 555555 bbS 5 S 5555 bb 555555 bbbb 55555555 
♦ 55555 b 5 b 5555 b 5 b 55 bb 5 bbSbb 555 b 55555 b 5353555 b 5555 bb 555 b 555 bbb 55 b 555555 


666 66666666666666666666666666666666666666666666666666 I 
666666666666666666666666666666666666666666666666666661 
6b666e66666666666666666666666666666666666666666666666* 

6b6666666666666666666666ei666666666666666ft66b666h66666I 
6*6 66 66b 6 6 66b666 666 666 66 666b6 66 66 66 66 6 6666666666666666 1 
66666666666666666606666666666666666666666666666666666+ 
666666666666666666666666666666666666666666666666666661 
66666666666666666666666666666666666666666666666666066! 
6666666666666666666666606666666666666666666666666666+ 
15 f) 55 bS 5 b 55 = 555 S 5 b 55 b 55 b 3 bbS 5555555553 b 3 bbb 55 b 5 b 35 bSbbS 55 bbb 5555 rj 55 b 5 b 6&666bb666666666666666666666666666A6666666666666666 I 
I 5555 b 355555555 b 3 b 5 b 5 S 555555 bbbbbbb 555 b 3 bb 5 bb 5555 b 5 bb 55555 bbSbS 55555555 6666 666666666666*66666666666666666666666666666666 I 

+ 555555 bbbb 5 bb 5 bSbbb 5 b 555 bb 5 Sb 555 b 5553 bbb 555 b 55 bb 5 bbb 555 s 5 bbS 555555555555 66b66666666666666666666666666666666666666666666+ 

I 555 S 53 bS 55 b 55 S 555 bbb 5555 b 555 b 5555555 S^ 33 b 5555 b 5 bbb 5 bbb 555 b 5 bS 55555 S 55 Sb 5 b 5 b 66666666666666666666666660666666666666666666! 

I 555555555 S 5 b 555 bbSbbbbS 55555555555555 bobb 5555 b 5555 bSb 5 bSbbb 5555555555 b 5 Sb 5555 66666666666666666666666666666666666666666! 

♦ 55555553555555 Dbbbbb 35 Sb 355 bSbS 5 b 5 b 55 bbb 555555 b 5 b 55 S 55 b 5 bbb 555555555555 bS 55 b 5 b 3 b 6 & 6666 A 66666666666606666666666666666 + 

I 5555 E 5 bb 55 bbb 5555 b 5 S 5 b 5 bbb 5555 S 5 b 55 b 3 bbb 5 b 55 S 55 bb 5355555 bbb 55555 ‘=; 55 bb 555 Sb 5 S 555555555 66666666666666660666666666666661 

I 55555 bbb 55 b 5 b 555 bbbbb 55 bb 5 b 555553 b 5555 bb 5 bbbb 5 bbbbbbb 555 bbb 5555 S 55555555 bS 5 b 555555 bbb 55355 6666666666666666S66666666I 

♦ 55555bbb555D55bbb5b5b55bb35555b5bb55Sbbb3555b5bb ^55,555555,555555,^555155 666666666666666666 + 

i555555b555555bbbb5bbb5bb55b5bb55b5555bb AAAAAAaAa A 4A AA444A44A44 A S555b555b5b55555555555 ' 66666666! 

1 555555355 55bbb555b3 5b b55555b5 44444 a444444444a444h4a444 444 44444444444444 5S5b555bBbbb5b5b5 555555b 65 I 

+55355bbb55bbb5bb55b5b 4444444444444444444444444444^44444444444444444444444444444 555555555555555655555555555355+ 

15555555355 44444 4 44^4 44 4 4444 4<*4 4444 444.*i, 444,44 4444444444444 55555555555555555555555555! 

IS555S 444444444444444444444444444444A44444 3333333 3 333 3353 333 333 33 44444444444 56555655555556555555^51 

♦ 4444444+44444444444444444444444444 333333333333^3333333333333333333333333 44444444444 5555555555555555 * 

I 44444444444444+H4+444444444444444 333333333333333.33333333 3333333333333 444444444444 55555555 44! 

I444444444444444444A44+44444444 33333333333333333333 22??22?222^22 3333333333 44444444444444 ' 444444I 

*44444444444444444444444+44 333333333333333 2?22222222222?2?22222??2222? 333333333 444444444444444444444444+4* 

X+44444+4444444444444444 3333333333333b 2222222222222222222???2222222?222?22222 333333333 4444444444444444444444441 

1+4444444444+444444A44 333 33 33 333333 22222222222222222222 22222222222222222222222222 333333333 44444444444444444444+441 

♦ 44444444444444444 333333333.3333 222222222222222222222 2222222222222 333333333 4444444444444444444444+ 

133 44444444 3333 133333333 22223222222222 1113 11 11 1 111111111 2222222222? 333333333 4444444444444444444441 

13333333 3333333333333333 222222222222 1 1 1 1 1 11 1 i 1 1 1 1 1 1 11 11 11 1 1 1 1 1 1 1 U 222222P2222 .333.333333 444444444444444444441 

+ 333333333333533333333333333333 ?2222?2a 2?2 1 1 1 i 1 1 1 1 1 1 1 1 1 1 i 1 1 1 11 H 11 H 1 1 1 1 1 1 1 1 1 22222222??? 333333333 4444444444444444444 + 

I3J33333333333333333 J3333333333 ?22222222? 1 1 1 1 1 H 1 1 1 1 1 1 1 1 i 1 1 1 : 1 1 1 1 U 1 1 1 1 1 1 1 1 1 1 2 1 2222?2???2 333333333 
I33333333j333J33333i3333J33335 2?222?2232 11111111111113111111)1111111111111111111 2?2222?222 33333333 
♦33333333333333333333333333333 2?2222?22 11111111111111111111111111111111111111111 22??22?222 333.33333 
13333333333333333333333333.333 2222222?2? 1)11111111111111111111111111111111111111 2222222222 333333333 
13333333333333333333 333333333 22?222??22 11 1 1 1 1 1 1 1 1 11 1 1 1 1 1 i 1 1 1 1 1 1 1 1 U 1 1 1 1 11 1 1 ] U 1 
+ 33333533533333533333333333.33 22?2222?2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ■ 1 1 1 n n 1 21 1 1 1 1 1 1 1 11 
1 333333333.333353333333333333 222?2222?2 llilllllllllllllllllUliniililiilllUlli 
I 33 333333333533333 33 J333333 3 22H?222??2 1 1 1 11 1 1 1 1 1 1 1 ' 1 11 1 1 U 1 1 1 2 l*i ) I ) 1 1 1 1 1 1 1 1 1 1 1 1 
+ 33333333333333335 333333333 22222222222 llllllillUllllllilllllUllllllllllllllll 
1 3333333333 3333333 333333333 222ii?22222 lllllliillllllllllllllinillllllllllllllll 
1333 3333333333 333333333333 2222222222 1 1 1 1 1 1 1 11 1 1 1 1 1 1 11 1 1 U 1 1 1 11 1 ) 11 1 1 1 11 i 1 1 1 1 ’ )1 

♦ 33333333333333333 J333333 ?22222?2222 111111 1111111 111111111111 1 11 1 11111 111 111)1 1 

13 335333333333 33333333 33 2222222222 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 

! 33333333333 3333333333 2222222222 lllllillllllllilllllllllllDHllllllillilllllll 


2222222P22 

H2???22??2 

? 2 ?? 222?22 

? 22222??22 

2222222P22 

?2?2222?22 

2 ? 2222 ? 2?2 

22 ? 2 ? 22??2 

? 2 ? 2222??2 

222222???? 


♦33333333333333333333 2222222222 1 1 11 1 1 1 1 1 1 1 i 1 1 1 1 1 1 1 11 1 1 i 1 J 1 1 1 1 1 ) ) 1 1 i 1 11 1 1 1 i 1 1 1 1 1 2?2222?222 
133333333333333333 333 22222222 llllllllillllllllillllllllllllllliiiniilliniiiiil 222222222 
13333333333333333333 222222 11111111111111111111111111111111111111111111111111111 222222222 
*333333333333333333 22222 1 1 1 1 1 111 1 1 1 1 1 1 1 1 11 1 1 1 1 11 1 11 1 1 1 11 1 U 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 11 22222222 


33633.3333 

3363333.33 
333 n.3333 
33 * 53.33333 
3333.33333 
33.3333333 

33333.3.3.33 

333333333 

33333333.3 

33.333333 

3*».33333 


4444444444444444444! 

4444444444444444444! 

4444444444444444444 + 
4444444444444444444! 
44444444444444444441 
4444444444444444444+ 
4444 4 444 4 4444444 + 44 ! 
444444444 + 444444444 I 

4444444444444444444 * 
44444444444444444 4 4l 

44444444444444444441 

4444444444444444444+ 

44444444444444444441 

44444444444444444441 

44444444444444444444 + 




1 


3 


6 9 


3,333333 444+44444444444444441 
333333 444444444444444444444! 
333333 444444444444444444444+ 

4. ^ 

10 11 12 


w 

o 

o 
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NOHTH DAKOTA DATA 1964 : MtTMOD 10 

9 +5Se:bbb5bb5bbb55 06666666666666666666666606666 77777777777 7 ' m 77777777777 ^77777777777777777777777777777777777777777777777* 
I5bb55bbi?3b5b5b5 066606666666666666600066666 7 !7 77 77 f 7 /7 7 777 77 77777777 7777777 T7 777 7 7777 7777777777 77 77777 77 7 77777777771 

I555b5&bb55bSb5Sbbb 6666666666660666606666666666b 777777777777777777777777777777777777777777777777777777777777777777771 

♦5555S5Sb55bb5bbbbbbb 66b666666666666to66666666666666 77777777777777777777777777777777777777777777777777777777777777 * 

I55555555b5b55555bS5555 6666666666666666b66666o66666666666b 777777777777777777777777777777777777777777777777777777771 

I5555555b5555555bS5b55555 666666666666666666666666666666666666666 777777777777777777777777777777777777777777777777771 

8 * 555555S55Sb555SbbS5S5555 6666666666666666666666666666066666606666666 77777777777777777777777777777777777777777777* 

144 555b5ob5S5bb55bbS5b5655 66666666666666666606666666666666666666666666666 777777777777777777777777777777777777771 

144444 55Sbb5bbbbb5555b55b55‘3 666666o66666b6666666666666o6666666666'66 6666666 66666 777777777777777777777777777777771 

♦44444444 G5b5555SbSbbb555S555B5 666606 66666606666 666666606 66666666666 0666666666 66666666 77777777777777777777777777* 

I <1444 444 444 5b55b55b555S55b55SSS5 66606666066066666666066666666666666666666666666666666666666 7777777777777777777771 

1 44444444 44444 SbbSSSbSbbBSBbSSBbbSb 06666666666666666660066666 666 66666666666 6666666666666666666666 77777777777777 7771 

7 +444444444444444 bSBB55BB5S5bS55555‘i8‘j 666666666o66b6660bb666666666666666666666666666666666666666666666 7777777777777+ 

144444444444444444 SbSSSbBSBSBBBBBBB'jBbB 660666666666666606666666666666666666666666666666666666666666666666 7777777771 

14444444444444444444 b5b5bS5b5b5BSb56BobbS 6666b066666666666666666666666666666666666666666666666666666 777771 

.444444444444444444444 obBSBbBSBbSSBBBbbbbBBBbBbbBBBB 666666666666666666666666666666666666666666666666666666 7* 

144444444444444444444444 55b5S55B5b55BbbbB5bB5BB5555b55BSbb555 666666666666666666666666666666666666666666666666666 I 

14444444444444444444444444 SbbSBBbSBSbbbbBBBbSBbbSBSbBSBBbbBBSBBBSBBB 60666666666666666666666666666666666666666666661 

6 *444 4444 44 444444 444444444444 55Bbb5BBo3655B555bb5bb5bb5bo6B5555Bb5b55 55556 66666666606666666666666666666666666666666* 

I 44<t444444444444444444444444 05555bb55555555bS55535b5bbbS55555555b555556bS555 066666066666666666666666666666666666 J 

1 333 444444444444444444444444444 b55bbs55555555b 55555555555555555565 555555555555555 66666660666666666666666666666601 

*33333 4444444444444444444444444444 555555555555555555555555555555555555555555555555555 06666660666666666666066666* 

13333333 44444444444444444444444444444+ 5555555555555555555555555555555555555555555555555555 6666666666666666666661 

1333333333 44444444444444444444444444444444 5555555555655555555555555555555555555555555555555555 66666666666666061 

5 +33333333333 444444444444444444444444+444444444 55S5b55555555555b5555b5555Sb5555S5555b5555555555555 666666666666+ 

133333333333333 444444444444444444444444 + 4444444444444 55555555555555555555555555555555555555555555555555 6666660 1 

I 3333333333333333 4+4444 444 4444444444 + 444444 4444 + 444 4 44 444 55555555555565555555555555555555555555555555555555 661 

*3333333333333333333 4444444444444444+4444444444444444444444444 555555555555555555555555555555555555555555555555555 * 

13333333333333333333333 444444444444444444444444444444444+444444444 55555555555555555555555555555555555555555555555551 

13333333333333333333333333 444444444+444444444444444444444444444444444 5555555555555555555555555555555555555555555551 

4 * 33333333333333333333333333333 4444444444444444444444444444444444444444444 5555555555555555555555555555555555555555* 

I 333 333 33333333333333 J33333333 3333 444 + 4444444 4444444 44444 44 44444 4 444 444 4 444 44 5555555555555555555555555555555555551 

133333333333333333333333333333333333333 44444444444444444444444444444444444444444 5555555555555555555555555555551 

♦33333333333333333333333333333333333333333333 444444444444444444444444444444444444444444 55555555555555555555555+ 

I 333333333333333333333 3333 33 333 33333 3333333 3 333333 4 + 44 + 44 + 4444444444444 + 4 444444444444444444444 5555555555555555 I 

I 3333333333333333333333333333333333333333333333333333 444444444444444444444444444444444444+44444444444 55555555551 

3 *22 333333333333333333333333333333333333333333333333333 4444444444444444444444444444444444444444444444+444 555555* 

1222222 333333333333 33333333333333333 33333 3333 333 333333 3 3333 44444444444444 + 4444 444444444444444 44444444444444444 51 

1222222222 333333333 33333333333333333 3 3333 3333333333 33333333333 444444444 4444444 44444 + 444 444 444 444 4444 4444444444444 44 I 

*222222222222 33333333333333 33333333333333333333 333333333333333 33 4444 44 4444 44444 44444444 44444 44444444444444 44444 44444 + 

12222222222222222 333333333333333333333333333333333333333333333333 44+444‘4444 44 44444444444444444444444444444444444441 

1 222222222222222222222 333333333333333333333333333333333333333333 444444444444444444444444444444444+4444444444441 

2 * 222222222222222222222222222222 33333333333333 + 3333333333333333 4444 444444 4 44 44 + 444444+4444444444 + 44444444* 

122222222222222222222222222222222222222222222 333333333J33333333333333333333 44444444444444444444444444444444441 

12222222222222222222222222222222222222222222222222 33333333333333333333333333333333333 44444444444+44444444444441 

*2222222222222222222222222222222222222222222222222222 333o333333333 3333333333333333333333333333 444444444444444444* 


12222222222222222222222222222222222222222222222222222222 ^333333 3.33333333.333333 3333333.333333.333333333.3 4444 444 444 441 
I22222222222222 222222222222222?222222s222a22222222222 22?2 22 3333333333333333333 333333335.3333333333333.3,3 333 3.3 44444 441 
+ 2222222222222222222222222222?22?222?22^;222222222222222222<:?2 3333.333333333333333333333333333.3333.33333.3333333333 44* 
11 22222222222222222222222222222?2222222222222222222222c22222 33333333333333333333333333333333.33333333333333333333 I 
1 11 1111 222222222222222222222222?22222222222222222222c2 2222222? 33333333333333333333333.333333333333333333333.3333331 
*111111111111 222222222222222222222222222222222?2??222222222???2??? 33333333333333333333.33333333333333333333333333* 
1 11 11 1111 11 1111 nil 22222222?2?2222222222222222222224:222222??2?222222 33.33333333333333333333333333333333333333331 
llinilllnilllllillllinil 2??222222222222222222222222222??2222222222??2 33333333333333333333333333333333333331 
♦llinilllllllllllllllllilinillUl 22222222222222222222222222222222222222222222 3333333333333333333333333333333* 


0 


2 


4 


7 8 9 in 11 12 


1 


3 


5 


6 
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NORTH OAKOTA DATA iVbS : 


KH.1H0D ' ( 

— * 4 - 




— — + — --+ — - + 


I ♦ ^44 «44.<» A44 444 44 A^<4 4 44<»4 5555‘jS’355t3“>5Sbb t>6 6666 66bDb6 6o66lJfet>oc&666fi666f>(S6 666 7777 777 7777777777 777777777 777 77777777777777 777 ^ 
I 44 4 44 4 4 444444 444 4 4 444 4 5S5f.b5!bb5ijSB^;6b 66 6666 666b6666 66 66tt,t;6666n6666o6666 7 7 77 77 7 7777 777777 V 77777777777 777777777777777771 
1 4 44444444 44 t 4444444444 bbSBbbSbbbbSSb 0666666666656 666666666 6666 666 6666 666 77777 7777777777777777777777777 7777777777777777 1 
+4444444444444444444444 5bb5bb5555b5S'S b66666b666b6fa6666666o666666666666666 7777777777777777777777777777777777777777777777+ 
144444444444444444444444 bSbbbbbbbbbbS 656666666666666666666666666666666666 7777777777777777777777777777777777777777777777 J 
I44444444 't 44444444444444 bbbbSbbbabbb 6666666666666666666660666666666666666 7777777777777777777777777777777777777777777777 j 
+ 44444444444444444444444 bSbbb-jSbbbbb 6066666666666666666660666666666666666 777777777777 777777 7777777777777777777777 7 77777 + 
14 444 4444 444 4 444444 444 44 Sbb55555bb5b 6666666666666666666660666666666666666 7 7 7777777 77777777 777777777777777777777777777771 
I 4444444 4 44444444444 44 44 bbbbbbbbbSb 66066666666666666666660666666666666666 '177 777 7 Tl 7 77777 7 f 7777777777 7777777777777 777777 I 
+ 444444444444444444 bSbbSbbbbbS 666656666666&66666666606b6666 666666666 7777777777777777777777777777777777777777777777+ 

133333 444444444444444 5b555b555 666&6b6666666666666666o66666666666666666 7777777777777777777777777777777777777777777771 

1333333333 4444444444444 55bb5Sb5b 666b6666665fa66666666666b666666666666666f 7 7777 7 777777777777777777777777777777777777777 1 

+33333333333 444444444444 5b55S55b b6666o666666666566666666o6666666fi66666666 777777777777777777777777777777777777777777777+ 
1333333333333 44444444444 SbbSSbbb 666 6bfa£,6b666b6b666b6666bo666666 6666666666 7777777777777777777777777777777777777777777771 
1 3333333333333 4444444 444 SSSbSbSb 666b6bAf 6666 66b66b 066660666 6666 5666666666 7 777777777777777777777777777777777777777777771 
+33333333333333 44444+444 bobobbSb 66666656666666666666660606666666666666666 777777777777777777777777777777777777777777777+ 
1333333333333333 444444444 SbbbSbb 6 666066666666666666666660 65666666666666666 777777777777777777777777777777777777777777771 
1333333333333333 444444444 5655565 666b66566666o6666566666bb666666666666666666 777777777777777777777777777777777777777 i 

♦333333333333333 444444444 5bb555ob e56bb56bbb66ob666666b66o66b666666666666666666 7777777777777777777777777777777 (:,ft+ 

133333333333333 444+444444 5b55bS5b 55bob566bbb6Qfobb5b666o6o66 666656666666 777777777777777777 6656561 

44444444444 55555655 6bc6666666b66666666666b6 35555555555 6566565666565565 666'6e665666l 

4444444444 + 4 bSbSSSobb Obb A 60666 66b 666 66 6660 55b555555b55b555 65656665666666556666666556666666666666665 + 

J ^S^5bbS55 b666b6o6ob66b566 053555555555 4 5b 66656666b66665666666566666666666b6666l 

5b5ob555b 666666666 55b5bs55b555 4444444 5555 65666b6566566665656666566666666666l 

*3J33j 33333 444444+4444+444444 o5bb5b55o5 6 55bb555b5bbS55 44444444444444 5555 6666665565666556665666666566665* 

444444+44444444444+ 5bb5555b555555 5b55bbb55555bb 44444444444444444444 55555 65666666666656666656656666651 

44444444444444444445 bSbSbOS bbS55555bbb5555 44444444444444+4444444444 55S555 666666656566565666656666661 

444444444444 + 444444444 025555555 = 55 444 + 4444444444444444444444444444 5555555 666666666666666666666666 + 

; 44444444 + + 4444444A44444 44444444444444 55555555 6665,6666666666666666b6 1 

1333333333333 44444444 + 444 4444444444 44 + 4 4 4 444444444444 444 444 44 44 44 4 4 44 4 44 4 444 4444 444444 555555555 666666666 666666666666 1 

♦333j33j33joJ333 444444444444444444+44444444+4444444444+44444444444 3 4444444444444444 5555S55S5 66666666666666666666+ 

+44444444444444444444444444444444444444444444 3 44444444444444444 555555555 666666666666666666661 

+ 4444444 + 444 + 444444444444 + 444444444444444 44444444444444444444 25555555 6666b6666666666b6666 1 

+3333333333333333333=3333333=3 44+4444444+444444444+4444 44 444444444444444444444 5555555 666666666666666666666+ 

. 444 + 4444444444 33^3 4444444444444444444+44 5555555 666 6666666666666 666661 

+ + 3333333333 444444444 + 44444444444 5555555 6666666666666666666661 

53jJ333333T:i3 44i^44AA/+444444444A4A4A4 S?bSS5 66A6^6ft6f»6o6666666666 + 
? 3233 = 3333333333533 333233333 3 33 = 33 33 3 3 333 333333333 333 4444444444444444444444 555555 6666666666666666666661 
1 _ 33= 333j- 3, 333=3-3 2=333333 3 3 3333333 3 3 =333333333 =3353333333333 33333333 4444444444444444444444 55555 66666666666666666666661 
+ 33333 333333 3 333333 33 3333333333 33 J 33 333JJ333333333333333 333333333. 3 3 333 44444444 4 + 44 + 444444444 555555 66666666666 6666666666 + 
I 333 3 33333 33 3333 33333333333333333333333333 3333333333333333333333333333 4444444444444444444444 555555 6666666666666666666661 
J^^^^^^^:?^^^'2^;]^^''^'^^‘^?^?:^^:^“^^333333332jp3333333p3333333333333 3 333 44 4 444 4 444 4444444444444 555555 6666666666666666666661 
3333333333 =33333-333-333-33333333 =3 3.33=j 3 33 3 3 33 =3 3 33 3 33 3333 33 3.J3T 3 33 4444444444444444+44^4^^ 5555555 666+6666666666666666+ 

, ^^^^♦^2J33333333=33323 3 =33333=3333333333 J333 333333 33333335333333333 4444444444444444444444+4 5555555 66666666666666666666 1 

44444444444444444444444 55'^555555 66666666666666666661 
+ 333333333333333333333333333333333=3333=33333333 = 3333333333333333333 444444444444444444 + 4444 555555555 6666666666666666666 + 
+ + 1333^333333333 = 3333333 4444 444444 4 44 4 4 44444 444 4 5555555555 666666666666666661 

! + + + + t + + + 44444444 + 4444444444444444 555555555555 6666666666 6666661 

♦3333333333333333-33333333333333333333333333333333333333333=333333 444444444444444+44444+444 bb'=;5555b55555 66666666666666+ 

I333333333333333333=3=3333333333333333J23,J333333333=3333333=33333 444+444444444444444444444 5555555555555555 6666666666661 

133333333333=333333333333333=333333333=22333333=33333333333=33333 444+4444444+4444444444444 5bS555555555555f?s5 66666666661 

♦333333333333333333=333=333333333333333=333333333=333233333=33333 4444+44+44444444+44444444 55555555555555555555 6666666+ 

I 3 3333333333333333333333333333. 3 .33 33333 = 2 = J33333 = = 3333333333 = 33 33 44 44 44 44 + 444 + 4 44 4 44444 4444 55555555555555555555555 66661 

13333333333333333333323333333333333333333333333 = 333 = 333333 = 3333 44 444 4 444 + 4444 444 44444 4+444 55555555555555555555555555 66 1 

♦3333333333333333333333333333=3333333333333333333333=3333333333 444444444444444444444444444 '?555555555S55555555555b55555 ♦ 
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NOHTH DAKOTA DATA 1^66 ! MtTMOD n 

9 +33 A b55&555S555555 66A6i^fefc6bA(>66666t>b66A6{j6bf)b6666AA666666666 A66666666666ft6(i666A 5'i5'S‘555555555555555 6fi6At,^ 

133 444444444444444 bSb5bb53bbJ>555 t566b6666ft666(3bDtobb6bfi6bo066fi6AA666A6666666fi6666666f.6b6A 5S55b5bb5555555b5b55 ] 

1333 44444444444444 bbSbSbbbbbbbbb bnbDb6j66fa66fab6b6666fi66boe,&b6666b666bb656bA6fif>6666 b555‘T55bf;b55555bbbS55bbb55b55555 3 

♦33333 44444444444444 bSbbbbbbbbSSb f.66bbfi6666b6bb66b6fo6o66u666b66b6bb6666666A66b SSbbbbSSBbbbSbbBSSbbbbSbbSbSbSbSbSSBb^ 

1333333 44444444444444 bbbSbbbbbbbbS 66bb6C6b6bbb0666b6b6b6b66666666b66b66b666 5555bSbS'=;55555S55SbS55555555555b5S5555555 3 

13333333 444444444444 55b55555b5555 bbb6A66b666bb666666666b66666bb66A66666 b‘jB5555555‘3bb55bbS55Sbb555'Sbb5'555b5b55555555 3 

8 +333333333 444444444444 bbbbbSbSBSbS bfabb46666f'6bb66666666bo6fa6b6ftft66b66 BbSBBSBSbSSBbSSSBbSBBSBSSSBbSSBbSSbSbBbSSSbSbSb* 
13333333333 444444 h** 4444 bSSSSbbSSbb 66bb6666b66fa66666b6666b66&666666 55555555b555b5b55B55555<555B555555B5SB555B555555b553 

133333333333 444444444444 bbbSbbbSbb 66bb(Sbb66ft6e66b666b6bbD666b66 bSbBSbBBSbSbBbSBSBbSSSSbSBbSbBbBSSSBBBSSBbSSbSSSSSbSB I 
+333333333333 44444444444 B5bS555bb5 bb6o(s66b*6666666b66b6bb6b66 BS5b555b5Bb5555555bB555B5555bS55555555B55BSb5b55555S55554 

13333333333333 44444444444 B5b5B55555 bbb(S6b6566bfa666b6b66b06 SBBSSSSBSbBBSBBBSbbSbSBbSBSbSbSBSSbbSSbbSbbBSBSSSSSSBSSsSS I 
133333333333333 4444444444 55Sbb55555 fao666666b66b66bb6bbbb 55555555<5b5b5555SS5BB5S55555B55555B5B>5bb‘55B5BS55S5b5b55555555 1 
7 +333333333333333 4444444444 55bS55555 6b6^,6b66666b66b666b ^55BbS‘;BS5555555bbB555‘5‘55b555b'ib5555B55S5B‘55SSB55S5555555b55555« 

13333333333333333 444444444 SSBSbbBSS fabbbbbbbbbbbbbbbb bSbSbBSSSbSBBbSbSSBSBSBBSbSSbbbSBbSbSSBSSBSSSbBBBBBBBSSSbSSSSSbSBl 
133333333333333333 444444444 555555555 bbAbbbbbbbbbbb 5bS5555555555b55S5b5555b55555bS55555555555555S5555«5555555555555555I 
+33333333333o33333 4444444444 55555555 obbbbbbbbbbb 555555bb55555S555555555S55555555S55555S55bS5bS5S5b55555555555S5555555 + 
1333333333333333333 4444444444 55555555 6666666 55555555b555S55555555555555555555555555555555555555555555S555S55S555555 I 

1333333333333333333 444^444444 55555555=5 555bbb53555sb55S5555S555555b55555555b5555S55S5555555555S555555555555555555 I 

6 T 33333 33333 33323 j33 4*t 4444 44444 53b5555=5b55b55b55b5555b5=bb5b555555555Sbbb55b55555555F5555bbbb55b5555555b55556bb55hb55555* 

I 333333333333333333 4444 44444444 •35555b5bbb5bb5bbbb5b5bbbbb555555bS55555b55555555b55555555S55555555S5555=;5!^5';5555555ro555I 

1 333333333333333333 44444444444444 555555b55bb5b55 = 55S55bbb555555';55555555555S5S555S55S55b555555555555b5555555555r55555555 I 

♦ 333333333333333 J33 4h 4 4 444 444444444 S5b555bbbb55bbbb55555b555555b55555555555555555555555b555555555555555555555555555555b+ 

13333333333333333333 44444444444444444 55555555555555 55=5555555555 5555555555555555555555555555555555555555555555555555 1 

I33333333333JJ33j333 444 4h4444444444444 4444 555‘'555555555555555''5555 5555 555 55555 1 

5 +3333333333333333333 4444. f 44 44 44444 44444 44444444444444444444444444444^444444 44444444 44 55555555555555555555^5555555 

I3333333333333333j33 444 444444444444444**44 4444 444444 4 444 444444 4444444 444 4 444 4 444 4444 44444444 5555555555 5 555555 555 555 55 5E; j 

13333333333533333333 +44 44 4 4 44444444 *-4 444444444444444444444^444444444444444444444444 44444444444 555555555555555555555= 55 T 

+333333333333333333 4444444444444444444+4^44444444444444444444444444444444444444444444444444444444 555555555555555555555+ 

1333333333333333333 444444444444444444444444444 44 + 444 + 4444444444444444444444 4444444444444444444444444 5555555555555555555 1 

1333333333333333333 4444444444+444444444444444444444444444444444444444444444 444444444444444444444444444 55555555555555555 1 

4 +33333333333333333 444 44 44444444444444 44444 4+4 444 4444444 444 4 44444 44444444444 4444 444 44444444 44444'444444444 5555555555555555+ 
13333333333333333 44444444+4444444444444444444444444^44444444444444444444444444444444444444444444444444444 555555555555551 

1 33333333333333 444444 + 444444444 I 444 + 4444444444444444444444444444444444444444444444444 55555555555551 

+3333333333333 44444444+4+4444 3333333333333333333333 444+444444444444444444444444444444444444444444444 555555555555+ 

133333333333 44444444444444 333333333333333333 333333333333 4444444444444444444444444444444444444444444444 55555555551 

13333333333 4 + 4444444444444 3333333333.3 = 333333333333333333333333 444444444444444444444444444444444444444444444 5555555551 

3 +333333333 4+4444+444444444 3333333333333333333333333333333J3333333 444444444444444444444+4+44444444+4444444444 5555555+ 

1333333333 4+44+444+444444 3333333333333=333333333333333333333.33333333 444+444444444444444444444444+4444444+44444 555551 

1333333333 4+4444+444+44+ 33333J333J333333333333333333 3333333333333333333 444+4444444+4444444+444444444444444444+44 5551 

♦ 333333333 444444444 + +44 333333333333333333333 33333333333333.3.333333333 44 + 4444444444+444444444 + 444444444+4444 + 4 + 

13333333333 4+444+44+44 33333333333333333333 222322 3333333333333333333333333 444444444444444444444444+444444+444444+41 

13333333333 4A44444+ 3333333333333333=333 2222222222 3333=3333333333333333333333 444444444+444444444444444444444444444+1 

2 +33333333333= =33333333333333333=33 222222222222 333 =33333333333.3333333333333 4++++++++++4+++++4+4+++++++++++++++4* 

I 3333333=33=33333==33J33=333=333333333==3 22222222222222 33 =33333333333333333333.333333 +++++++4++++4++++4++4+++++++4+++4+ j 

133333=3333333333=3333333333333333=333== 2222222222222222 3=333333333333=333=3333333=333 4444444444444444444+4444444444441 

+33333333333333335=33333=3333333333333= 222222222222222222 =3333333.33333=333=333333333333'’ 44444444444444444444+444444444+ 

13333333333333333=33333333333333333333 22222222222222222222 3333333333333333333333333333.333 44444444444444+44444444444441 

1 333333333333 = 3333 3333333333333333333 22222222222222222222222 3333 33333333333333333333333’333,3 4 444444444444444 444 444 444 4+1 
1 +3333333 = 333333333 3333=33333333 = 333 2222222? 2 2? 2222222 22? 2222 333.3 3 533333. 33 33 3333333333 ’33333 44 444444 444444 444444 44444* 
13333333 33 33333333 333333333 3333333 2P?2222222222222222222222222? 3.3333333333333333333333333333333 44444 + 4444++ 444444 444 44 1 

13333333333333333=33333333333333 222?222222222222222222222222?22 33333333333.333333333333333333333 44444444+4444444444441 

+3333333333333333333=333333333 2222??222222222222222222222222222?2 33333333333333333333333333333333 4444444444444444444+ 

1333333333333333333333333333 222222??222222222222222222222222222???2 33333333333333333333333333333333 444444444444444441 

1333333333333333=333333333 222222222?2222222222222222222222222222?22?2 333333333333=33333333.33333333333 44444444444444+1 

0 +33333333333=333=3333=33 22222222222222222222222222222222222222222222222 33333333333333333333333333333333 444+4444444++ 

0 1 2 3 + 5 6 7 B 9 10 11 12 

V*> 
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5 ♦555555555555555bS5b5b55Sb55‘j55555b5S'5b5SSb55»5Sbb55£.555555b5b'i5SS'??<iS5 666b666666666666666666f>66ft6666666666666f>6666666666 + 
I 555555b55bb5bbbSbb5bbb5bbb55b'i5bS5b5bbbbS555bSbbb55bb55bbbb555bbbbb5bb 666A 66 6666666 b66666666''>f)666 6666666 66 6^6666 6666666 6 1 
1 555525bb5b5b555bbbsDbb5b3555bbb*>b5'35b^‘">5S5bbbbbbb5bb3b55bbS555b553b5b5 666666666666666666666666666666666666666666666666661 
+5b555535555bb55bbbD5b555S5b555555b555b63b555b55b55bS35‘55bbSi5b5bS553565 66666666666666666666666666666666666666666666666566+ 
ISSSbbSSbSSbbbSbbSSbSSSSSbSSbbbSSSbSbSbbbbbSbSSbbSbbbbbbSSSbSSbSSSbSSbb 666666666666666666666666666666666666666666666666661 
I 55535 b5bS5S5b5bbbbb 55 55 5bbbb55S55b5535!3&')b55S5Sbbb5555555bb 55556 666555 666666666666666666666666666666666666666666666666661 

8 ■►55b555555555555bbSb5bb5b555b5555b5555bbb65555b55b555s5555Sb55555555555 66666666666666666666666666666666666666666666666666* 
I55555555555S5S55bSb5b5555555S555555555b5555555555bb555555bbb55bb555555 666666666666666666666666666666066666666666666666661 
IS555Sbb555555SSb5b55Sb5b5555555555555bb555555555555bb55555b5555b555555 666666666666666666666666666666666666666666666666601 
*5SS555bb5555555b5bb55b5b555b5b555b5S5Sb555555b55b55b55555bb5555S5555S5 66666666666606666666666666666666666666666666666666+ 
I555555b565555555b5b55b555b5555555bb555555555S5bb5555b55555b55555>555555 666666666666666666666666666666666666666666666666661 
I355S555555553Sb555bbbbbbb55b5555b5b55bb25555555555bbb555b5355555555555 66666 666 66666666666666666066 666 66 66666 66 66 66666 66A1 
7 *55556565 5555 b5S5b5b5bb5bb55555555b5555b 55 555555b55b555b5bbb55bb555555b 66666666666666666666666666666666666666666666606666+ 
! 5 55555Sb5bb55555bb5bbb5 355555 5S5b5555b3b 55555Sb5635bb5b55 bb5S555555555 666 6 66 66 66666 66 66 66666666 66666 6666 66 66666666 666666 1 
IS5b555b555bS5b535bbbb5b5b55555555555S5bb-3S555SbbbS555555b5b5555S555555 666666666666666666666660666666666666666666666666661 

♦ 555 355555 555b55bb 5555 555 555 5555555 555bbb5b5b5S5555bb 555 555555555555555 66666666666666666066666666666666666666666666666666 + 
! 5555555555555555555555565555555555555565555555555555555555355555555555 666666666666666666066666666666666666666666666666661 
: 5355555555535553535555535555555555555555555555555555555555555555555555 666666666666666666666666666666666066666666666666661 

6 *5553555555555535655335533555535555555533555555555555555553355555555555 6666666656666656666666bb6666666660.66666 66666606666* 

1 555 5S53 3553b3bb353 55b35535S55b55b555553Dt3D5b35b 3555533 5555 355555555555 66066666666666660666666666606666666666666666666661 

: 55 535533555355535555555555555555555555355 555 5655555555555555553555555555 666066666b 666666666666666666666666666666666666661 
+55555 55 555555555355 56 553555 5555555555355=;555555 55555555555355555555 5 666666666666666666666666666666666666666066666+ 

155555555555555555555555555555555555555555555555555555555553555555 44 3 44 66666666666666666666666666666666666666666661 

1555555555553555555555555555555555555553555555555555555555535555 4 33 3 4 4 666666666666666666666666661 

5 +5555555555555555555555555555555555555555555555555555555553555 44 33 223 44444444444444 555 + 

!55555555555355555555b5555555555555555b55 4 3 222222? 3 44 555555555555555555555551 

15555555555555555555 4444444444444444444 33? ?2? 22 2222222222222 3333 44 I 

* 44444444444444444 333 3 2 2222 lUH 22222222222222 333 4444444444444444444444444* 

I4^-44:,44444444444444444444444444 33 2222222222222222222 2?2 llllllill 2222 333 4444444444444444444444441 

14444,444444444444444444444 3333 2222222222222222222 1 1113111111111111111111111 2??2 33 4444444444444444444444441 

4 ♦* 33333333 ?? I iiniiiiniiiiiiiiiniiniiiuii 222 33 444444444444444444444444+ 

! 33333333333333333333333333333333 22 1111111111111111111111111111111111111111111111111111 2?2 33 4444444^44444444444444441 
I23333333333333333333J3333333J3333 22 1111111111111111111111111111111111111111111)11111111 222 33 4444444444444444444444441 
*33333333333333333333333333333333 222 1111111111111111111111111111111111111111111111111111 222 33 444444444444444444444444+ 
133333333333333333333333333333333 222 1111111111111111111111111111111111111111111111111111 222 33 4444444444444444444444441 
!333J3333333333333333333*333333333 22 11111111111111111111111111111111111111111111111111111 2?2 33 4444444444444444444*44441 
3 *33333333333333333333333333333333 22 11111111111111111111111111111111111111111111111111111 2?2 33 444444444444444444444444+ 
:3-3333333333333i3333333333333333 22 1 1 1 i 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 il 1 1 1 1 1 1 1 1 1 1 1 1 1 I 111 1 1 1 1 1 1 1 1 1 11 222 33 4444444444444444444444441 

133333533333333333333333333333333 22 1 1 1 1 1 1 1 1 1 1 1 1 i 1 1 i 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 L 1 1 1 1 1 1 1 1 1 1 1 1 11 222 33 444444444444444444444444 1 

*33333333333333333333333333333333 22 111111111111111111111111111111111111111111111111111.11 222 33 444444444444444444444444* 
:3333333333333333333J333333333333 22 11111111111111111111111111111111111111111111111111111 2?2 33 4444444444444444444444441 
! :-'333333333333333333333333333 l33 22 111111111111111111 1 111 1111111 111111111111 111111111 111 222 33 4444444A4444444444444444 1 

2 *53333333333333333333333333333333 22 3 1 U 11 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 ] 1 1 1 1 1 1 1 1 ! 1 1 1 11 1 1 1 1 1 1 1 1 .1 2?2 33 44444444 44444444 4 44 444*4* 

133333333333333333333333333333333 22 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 i 1 1 1 i i 1 1 11 1 1 1 1 1 U 1 1 11 1 ! 1 1 1 1 1 11 1 11 1 1 1 1 2?2 33 44444444444444444*444444 1 

1 3323333333J3333333JJ3333J3333333 22 11111111111111111111111111111111111111111111111111111 2?2 33 4444444444444444444444*41 
*33333333333333333333333333333333 22 11111111111111111111111111111111111111111111111111111 2?2 33 444444444444444444444444* 
133333333333333333333333333333333 22 11111111111111111111111111111111111111111111111111111 222 33 44444444444444444444444*1 
13'3333333333333333333333J333335 22 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 U 1 1 1 1 1 11 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 11 11 1 1 ?222 33 444444444444444444444*441 
1 *3333333^3333333333333333333333 ?2 111111111111111111111111111111111111111111111111111111 2?22 33 444444444444444444444444* 
i 333 :3 13333333333333333333333 22 1 1 1 1 1 X 1 1 1 11 1 1 i 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2222 '33 *44444444444444*444444441 

133333333333333333333333-1333 222 11 1 1 1 1 1 i 1 1 11 1 i 1 1 1 1 1 1 1 1 1 1 1 U 1 1 1 1 1 1 1 i 1 1 1 1 1 1 1 1 1 i 1 1 1 1 1 1 11 1 1 22222 333 4444444444444444444*444 1 
♦3333333333333333333333333 222 1111111111111111111111111111111111111111X11111111111111 2?2222 333 4444*4*4444444444444444+ 

I33333333333333333333J333 222 11111111111111111111111111111111111111111111111111111111 22222 33333 444444444444444*4444441 

13333333333333333333333 22 111111111111111111111111111111111111111111111111111111111 22?2?2 33-3,33 44444444444444444444441 

0 *333333333333333j3333 22 1 1 1 11 1 1 1 1 1 1 1 U 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 X 1 1 1 11 1 1 1 1 IT 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 222222 33333 44444444444444444*4444 + 

0123456789 10 11 12 
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HOD 'i 
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*nmmmiminmnm m ninm'm'iTnnniimi 11 1 ’( / 777 77 77777 7777777777777777777777777777777777777777777777777777* 
immnmimm n nm unnm niinmiini n rmnnnimmminmmmi nr rmnnmimmmm nmm nii 
mini im n nnmn n mil n 111 nm mimn m nil mil 11111 riiiiii''iiiiiiiniiiiiimninm nniiimiiiiinimm 

+ 7777777777777777 ?7 77 7/7777777777777777777777777 11111 ri 777777777777777 777777777777 777777777/77 77777777777777777777777777 7 7* 
I 7777777777/77777777777777777///777/777777777777//7777777777777777777777777777777777T777777777777777777777777777] 

16666666666 777777777777777777777777777777/7777777777777777/77777777777777777777/7777777777777/77777777773 

+6666666o6cb6666bbo6b6666b666 '^77777777777/77777777777777777777777777777777777777777777777777777777777/77* 

I66666b6fa66bb66666666bfa6666666666666666b6bbbbbb 1111111111711171111111111111111111117111117111111111711111 

I6666D6666666bb66666666666666666666fa666b66b66666o6666666666b66666 1111111 11111111111111111111111111111111 

♦ 6b6666bb6b66666666666b666666666666666bob666666666666666666b666666666666666666666666 11111111111111111111 * 

I66666666666666666fa6b666b6666666666666fe0b6666666666666666bb06b666666666666b666666666666666666666666666 771 

I6666b666666b6bb6666b6b66b66666666b666bobb6666b66666666b666o666666666666666666666666666666bb666666666666666666666666666b I 

* c. . -c u - ,^‘’^‘’‘’*'’^^^^^'^^^*^^^-'^‘'‘'°^^^^*=^^‘’^^^^^^&^^^‘’ 6 ^^* 6 n 66666666666 b 666666666 b 666 (> 666666666666666666666 b 66666 b 6 b* 

* - 666bb66666 6b6b66bb666bfao66 66666666b666666666&666666666b666666b666666666666566666666bbbI 

^ = = ”bbb = 5b5bS_ 55655555535^)56 6666<5b66 666A66666666b66 6666666b666666666b6666666666666666b666666 6666I 

= 666666666666666b6666666666666666666666666666666b66 + 

I555555bb5555b55b55S5D5Sb5B5b5555bb555bb5s56Sb5555555555S55b555555555S55 6666f>66666666666666666666666666l 

I5S5555bb5555b555b5b5S55b55555b55555555b5555555b555b55555556555555555555S555555555555555555 6666666666666! 

+ 55555555S55bb55555556555S5B5BB6^55B^AbbbBR^Ab^B5'-^'^BA^bUt^^^Sc;c:clRRc;cccc:cuc.c;^xc;c^;c:^ic:ctl^c;I:.etc^pc•ce?cc!M(r/-^^ff?^-f-tr-l-•-^^r ^ 


^ ^ - J555555555555555I 

l ‘*‘*‘*‘*‘* 3 55555b555555bbbb555b5b555555S 55556 5555 55555555555555 55555555555555555555555555555555555555555555 I 

+ ^S^SSSbSSSSbBSSSbbSbbSBBBSSSSbSSSSSBBSSBSSSBBbSbSbSSSBBSBSSBBSBBBSBBBBBSSBbBABijS ♦ 

1 444444A4'.4*, 4444 444444444444444444444444444 555555555555555555555555555555555555555555555555556555555555 51 

I 44 4 4 44 44 444 44 4 4 44 4 4444 4 444 4 44444 44 44 4 444 4 4 4444444<t 444 4444 4 44 5555555555555555565555555555555555555555555 1 

+444444444444444444444444444444444444444444444444444444444444444444444444444444 555555555555555555555555+ 

1 4444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444 5555551 

14444444444444444444444444*44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444 ^ I 

* ^■^'*''’^^^‘+^'*‘ii4444*4444444444444 4 44444444444 444 444444444 44*4 444444 44444 444 4444444 444444444444444444 444 44 4 44 4444 4 + 

1333333333333^ _ .... . ^‘^^'*‘^444444444 4444444444444444 4444 44 4 4 4’44 444 4444 4 444 4444 44444444444 444444444 4444444 44 44 4 44 T 

I3333333J333333333j3333333333333 A4444 4 444 4 44444444 4 44444444 44444444 4 444444444444444*4 4 4444444444444444441 

‘*'^33^j3i33-53j333j33b3333b3333b333333333'^'^333333333 444444444444444444444*44444444444444444444444444444444 + 

1 3333333333533333333333333333333333333333333333333333333333333333333 44444444444 44444444444444444444444 44 T 

1 3 333333333333 333333 J33333333333333333333333333333333333333J33333.333333333333333333333 . 444 44444444444 444 T 

+ 3 33333333333333333333333J33333J3333333b3 3333 3333 3333333333333333 3333333333333333333333333333333333333333 ♦ 

I 3333333333333333333333333333333333333J3 T333 3333333 3333333333333 3333 333 3333333 3333333333333333333333333333333333333333 333 I 

^^^'^^^-'^■^^^^^^^^•'^33-^-^^-*^'‘333333353333333J3333333333333333333333333333333333333333333333333333333333333-I 
* 2 ? 2 ZZd ? aZZZdZci 22 Z?.d '*•^333333333333333333^33333333333333333333333333333333333333333333333333333333333 333 + 

. . ^'*■*33333333333333333333333333333333333333333 3333 33333333 35333333 31 

I2H2c:2324 2222</2cJ2s2225//2222222?22222?2‘:!‘^</2??2222232<'22??22 33 3 3333333 3333 333 33 33 33 333 •^3 33 ^3■^33■;333■^■^T 

!?222222222222222222222222222222?2222222222222222222222222^222222?222222222 " 3333333333333333333333331^3331 

33333333331 

>?222222222222222222222222222222222?2?22?22?222 I 

, , -22222222-i22P22222222222222222222?22222??222?22?222222?2?2222??22?22222222222222 + 

J m , 222222222?2242222222222222222222222222?222222?22?2222222222222222222?2?222222222?2?22222222222?I 
fi )) M , , , ,, ,,, , ) , 2222222222222422222222222222222222222222222222222222222222222222222222222222I 

2??222222222222?222?22222?22222222?22222222222222222222222 + 

nninniniiiinniniiiiuiiiiinnniininuiniiiiiiiiiuniiiiuiiiininii 2222222122^22^21^ 

+ 111111111 iinimiinuniiiiimiiiimiimiiniiiiiiniii 111 inmiiiuin minimi iiiiiimii ^^^^^^^^^^^^ 222* 

iimimimmiiiiiiimmmiiimmmmimiimmmmmmmiiimimmmmmmmimmmimm i 

♦ 000000000000000 iinmimmmmmmiimmmmmmmimmmmmmiimimmmmn* 

lonooooooooooooooooooooooooooooooo iiimiimmmmuimmmmmmummiiiimmimmmi 

loooooooooooooooooooooooooooooooooooooooooooooooooooo iiiiiiimiiiiiimmiiiiiiiiiiinniiiiiiiiniiiii 

♦OOOOOOOOOOOOCOOOOOOOOOOOOOOOOOOOOOOOOUOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO 111111111111111111111111111111111+ 

+ — — * — * * — — * — — * — * — 

0 1 2 3 4 5 6 7 8 9 \0 U j2 



ro 

o 

vn 
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COMPUTER PROGRAM DESCRIPTIONS 

A number of Fortran Computer Programs were written during the 
completion of this contract. Rather than include all of the corresponding 
listings (approximately 50 pages), this section contains a list of the 
programs written by The University of Texas at Dallas along with the correspond- 
ing input and output.* Computer decks for any of these programs can be obtained 
upon request. 

1.0 OBJECT 

Purpose: Perfom variational analysis using low-pass filtering 

constraints as outlined by Wagner C t1 • 

Input: (l) Coordinates and values of data points. 

(2) Coordinates of check points. 

Output: (l) A complete grid of extrapolated yield values after the 

completion of Wagner' s ite rative procedure. 

(2) A separate list of the yield values at the specified check 
points . 

2.0 METH09 

Purpose: Determine MBE's after Steps 1 and 2 using Method 9 as proposed 

by Shepard [163. Different values of the various parameters 
are considered. 

*The descriptions are brief. For a more detailed program description of 
any of these routines, contact The University of Texas at Dallas. 



Input: 


Output : 

3.0 METHLO 
Purpose : 

input : 
>utput : 


4.0 EXTMP 
Purpose: 

Input : 


(1) Coordinates and values of the data points. 

(2) Parameter ranges to he considered. 

(3) Coordinates and actual values for check points. 

For -each set of parameter values, the output consists of 
the MSB's and average absolute deviations for (l) Method 6 
(2) Method 9 after Step 1 (3) Method 9 after Step 2, 

Determine MSB's for each of the four models using Method 9 as 
proposed by McLain ^2] and discussed in [5]. For each model, 
four weighting factors are considered. 

(1) Coordinates and values of the data points. 

(2) Coordinates and actual values for check points. 

For each weighting factor, a one page listing is obtained 
giving the actual vs. approximated value using each of the 
four models at each check point. Also determined are the 
MSB's and average absolute deviation for each model. 

Calculate and display the results of applying Methods 1 
through 8 to a set of yield data as described in [5l 

(1) Number of check points and data points. 

(2) Check point coordinates. 

(3) Data point coordinates and values. 

(4) Actual values for check points. 

(5) Method 8 (object) values for check points. 
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Output: Actual vs. estimated yield values for each check point 

and each method. Included also are the MSB's and average 
absolute deviation for each method. 

4.1 EEGION (part of EXTRAP) 

Purpose; Determine if each of a set of input points lies within 
the convex hull of a set of data points (input). 

Input: (l) Coordinates of data points. 

(2) Coordinates of points to be checked. 

Output : A vector of zero ' s and one ' s for each check point where 

a one indicates that the point lies within the convex 
hull of the data. 

4.2 COIffilH (part of EXTRAP) 

Purpose: Determine the matrix of n objects using 3 at a time. 

Input : n 

Output; Corresponding matrix, e.g. for n = 4 , the output would be 

123 
12 4 

134 
.234 

5.0 COMAP 

Purpose: Plot a contour map for a grid of data. 

Input: (1) The ranges for each of the 9 symbols in the map, 

'e.g. 1 = 15-17.5, 2 = 17.5-20, etc.' 

(2) The value at each grid point. 

A contour map. 


Output : 
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- 6.0 QUADBl 

Purpose: Calculate the proportion vector p for the equation Ip = e 

as discussed in Cul . This routine uses a modified simplex 
method to determine p. 

Input: Matrices P and e 

Output: Value of p 

7.0 BIELSPM 

Purpose: Same as 6.0 except this routine uses the procedure outlined 

by Nelson a. 

Input: Matrices P and e 

Output: Value of p given by 

P = a + By 

where (l) y is an arbitrary vector 

(2) a is an output vector 

(3) B is an output matrix 

7-1 PSEUDO (part of NELSPM) 

Purpose : Determine the pseudoinverse of a matrix, A. 

Input : A 

Output': Pseudoinverse of A = A 

8.0 .MISCLS 

Purpose: To plot probabilities of misclassification as discussed in W- 

■ Input: (l) a priori probability of TT]_ = p^ 

(2) Values for ~ 

■" Output: CAICOMP plots of 


( 1 ) P (1I2) 

(2) P (2I1) 

(3) Total probability of misclassification 
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